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FOREWORD 


The  U.S.  Army  Research  Institute  (ARI)  helps  the  Army  with 
its  training  mission  in  a  variety  of  ways.  Evaluating  techno¬ 
logically  based  systems  designed  to  assist  training  is  one  area 
in  which  ARI  contributes.  The  Presidio  of  Monterey  Field  Unit  is 
colocated  with  the  Defense  Language  Institute  (DLI) ,  which  has 
the  mission  of  training  linguists  for  the  military.  When  DLI 
sought  to  exploit  interactive  videodisc  (IVD)  technology  in 
teaching  language,  ARI  agreed  to  assist  by  evaluating  an  IVD 
system  designed  for  the  German  Gateway  program.  The  research  was 
performed  under  an  existing  Letter  of  Agreement  between  DLI  and 
ARI  entitled  "Cooperative  Research  Program,"  dated  April  1984. 

The  results  of  the  research  were  briefed  to  the  Chief  of  the 
Educational  Technology  Division  of  DLI  during  August  1987.  The 
findings  are  being  used  not  only  by  DLI  in  the  training  of  lan¬ 
guages  but  also  by  a  broader  community  interested  in  improving 
training  through  technological  innovation. 
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EFFECTIVENESS  OF  THE  INTERACTIVE  VIDEODISC-ENHANCED  GERMAN 
GATEWAY  PROGRAM 


EXECUTIVE  SUMMARY 


Requirement: 

Integration  of  emerging  instructional  technologies  into 
existing  training  programs  has  the  goal  of  improving  training 
while  reducing  costs.  Evaluation  of  new  systems  in  the  context 
of  the  training  program  is  the  best  method  for  determining  if 
this  goal  is  being  achieved. 

The  Defense  Language  Institute  (DLI)  developed  an  inter¬ 
active  videodisc  (IVD)  system  for  use  in  German  Gateway  course. 
The  Army  Research  Institute  (ARI)  conducted  an  evaluation  of  the 
IVD  system  in  a  field  test.  The  objectives  of  the  research  were 
(1)  to  compare  the  effectiveness  and  acceptability  of  the  video- 
enhanced  German  Gateway  program  with  the  existing  German  Gateway 
program  and  (2)  to  identify  user  issues  in  the  adoption  of  the 
interactive  video  course  materials. 


Procedure : 

The  research  used  a  Baseline  group  of  49  students  who  par¬ 
ticipated  in  the  existing  course  in  1984-5  and  an  IVD  Field  Test 
group  of  40  students  who  took  the  video-enhanced  course  in  1986. 
Group  comparability  was  assessed  for  demographic  and  German- 
language-experience  measures.  Students'  German  proficiency  and 
attitudes  about  the  course  were  compared.  Language  proficiency 
was  measured  both  by  self-report  and  by  pairs  of  trained  raters 
in  an  Oral  Proficiency  Examination. 


Findings: 

The  two  groups  were  comparable  on  precourse  measures  with 
the  exception  of  time  in  Germany:  The  IVD  Field  Test  group  had 
significantly  more  time  in  Germany.  Controlling  for  this  factor; 
proficiency  in  German  was  greater  for  the  IVD  Field  Test  group. 
However,  the  final  examination  for  the  course  was  changed  at  the 
same  time  as  the  introduction  into  the  course  of  the  IVD  mate¬ 
rials  and  probably  contributed  to  the  effect.  Listening  compre¬ 
hension  of  German  was  also  better  for  the  IVD  Field  Test  group 
than  for  the  Baseline  group.  The  IVD  materials  were  well  re¬ 
ceived  by  both  students  and  instructors,  although  there  was  some 
ambiguity  about  the  role  of  the  materials  in  the  German  Gateway 
course . 


vii 


Utilization  of  Findings: 

The  findings  from  this  research  have  three  different  levels 
of  use:  (1)  they  can  contribute  to  the  future  role  that  the  IVD 
materials  developed  for  the  DLI  German  Gateway  course  will  have 
in  that  program;  (2)  they  can  provide  insight  into  the  use  of  IVD 
technology  in  language  training;  and  (3)  they  can  benefit  the 
more  general  application  of  IVD  technology  to  training. 
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EFFECTIVENESS  OF  THE  INTERACTIVE  VIDEODISC- 
ENHANCED  GERMAN  GATEWAY  PROGRAM 


INTRODUCTION 

The  U.S.  Military  operates  in  a  worldwide,  multilingual, 
multicultural  context.  The  ramifications  of  this  fact  on  lin¬ 
guistic  training  requirements  present  a  formidable  challenge.  A 
consistent  need  exists  for  a  vast  and  constantly  replenished  sup¬ 
ply  of  linguists  trained  in  a  wide  range  of  languages.  The  re¬ 
quirement  for  training  includes  the  organizational  capacity  to 
teach  even  infrequently  needed  languages  as  well  as  the  ability 
to  respond  to  the  sudden  and  often  unanticipated  surges  in  demand 
for  a  particular  language.  The  Defense  Language  Institute  (DLI) , 
which  has  the  foreign  language  training  mission  for  the  military, 
has  found,  despite  claims  made  for  various  methods — from  Berlitz 
to  Superlearning — that  learning  a  foreign  language  is  a  highly 
time-intensive  process.  Yet,  the  military  must  rapidly  train 
linguists  to  functional  levels  of  proficiency  to  achieve  maximum 
benefit  from  short-term  enlistments,  as  well  as  to  meet  surge 
requirements. 

The  contemporary  focus  in  foreign  language  education  on  the, 
so-called,  communicative  approach  emphasizes  speaking  and  listen¬ 
ing  skills  in  realistic  and  authentic  communicative  contexts 
(Krashen,  1982;  Sanders  and  Kenner,  1984).  This  approach,  how¬ 
ever,  has  generated  limited  development  of  technological  applica¬ 
tions  to  the  field  (Underwood,  1984) .  For  example,  computers 
seem  more  readily  adaptable  to  the  kinds  of  learning  activities 
associated  with  traditional  methods  emphasizing  grammar  and 
vocabulary  drills.  Indeed,  one  argument  for  the  development  and 
use  of  foreign  language  Computer  Assisted  Instruction/Computer 
Assisted  Learning  (CAI/CAL)  programs  is  that  they  can  free  the 
instructor  to  conduct  communicative  activities. 

With  the  development  of  interactive  video  based  upon  the 
technology  of  interactive  videodisc  (IVD),  however,  comes  the 
capability  of  providing  the  student  precisely  the  realistic  and 
authentic  context  the  communicative  approach  prescribes.  Fur¬ 
ther,  the  use  of  interactive  video  in  foreign  language  instruc¬ 
tion  has  high  face  validity.  Not  only  is  learning  enhancement 
provided  by  multisensory  presentation  of  instructional  materials, 
but  also  interactive  video  has  the  capability  to  present  a  lan¬ 
guage  in  realistic  situational  and  cultural  contexts  where  the 
student's  ability  to  understand  and  respond  results  in  simulated 
real-world  consequences.  In  addition,  a  machine-based  instruc¬ 
tional  system  affords  a  versatile  and  flexible  means  of  augment¬ 
ing  existing  teaching  methods  and  curricula,  thereby  potentially 
increasing  institutional  flexibility.  Verano  (1987)  presents  an 
excellent  literature  review  and  experimental  results  indicating 
that  interaction  with  the  learner  is  the  critical  feature  of  IVD 
technology  in  achieving  superior  learning. 
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Background 

To  take  advantage  of  potential  technological  contributions  to 
training  capacity,  DLI  established  a  specialized  office  to 
examine  innovative  instructional  technologies.  The  Educational 
Technology  Division  of  DLI  recognized  the  potential  of  IVD  in 
foreign  language  education  and  began  to  explore  its  developmental 
feasibility.  This  effort  became  a  complex  array  of  individuals 
and  organizations  with  the  unifying  objective  of  developing  and 
testing  the  interactive  video  technology.  The  German  Gateway 
Course  became  the  testbed  for  the  evaluation  of  the  technology  at 
DLI. 


The  Monterey  Field  Unit  of  the  Army  Research  Institute  (ARI) 
agreed  to  assist  DLI  in  the  evaluation  of  the  video  enhancement 
to  the  German  Gateway  Program.  In  consultation  with  the  New 
Systems  Training  Office  and  German  Gateway  staff,  ARI  developed 
an  evaluation  plan  which  guided  evaluation  activities.  The 
Research  Branch  at  DLI  assisted  in  refining  and  defending  the 
plan. 

German  Gateway  program 

German  Gateway  exists  to  provide  brief,  intensive  training  in 
German  language  and  culture  to  Army  officers,  primarily  at  the  05 
and  06  levels,  assigned  to  command  positions  in  Germany.  The 
rationale  for  the  course  is  both  political  and  functional  in  the 
sense  that  a  functioning  command  of  the  language  is  neither 
required  by  these  officers'  positions  and  duties,  nor  is  it 
attainable  (beyond  level  1)  in  the  time  available.  Rather,  the 
purpose  of  the  course  is  to  encourage  interaction  between  the 
officers  and  the  host  German  communities  through  the  development 
of  cultural  awarene33  and  sensitivity  and  the  provision  of  a 
minimal  language  ability. 

The  Gateway  course  consists  of  sixteen  modules,  eleven  of 
which  must  be  completed  and  a  final  exam  taken  to  receive  a 
diploma.  The  final  exam  grade  and  number  of  completed  modules  Is 
recorded  on  the  officer's  record.  Each  module  consists  of  a  set 
of  audio  cassette  tapes,  text,  grammar  workbooks,  self- 
administered  quizzes,  and  cultural  information.  The  officers 
spend  6  to  8  hours  a  day  at  the  course  facility  and  many  study 
during  evenings  and  weekends . 

Instructors  are  available  to  work  with  each  student  on  an 
individual  basis  for  at  least  an  hour  a  day,  and  students  are 
expected  to  avail  themselves  of  this  resource.  The  results  of  a 
German  Gateway  program  needs  assessment,  conducted  prior  to  the 
commencement  of  IVD  development  for  the  course,  indicated  that 
instructors  and  students  alike  regard  the  time  spent  with  the 
Instructors  as  the  most  important  educational  element  of  the 
course. 
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The  course  is  characterized  as  self-paced,  but  the  students 
quickly  learn,  with  guidance  from  the  instructors,  that  they  must 
move  along  at  a  fairly  standard  rate  if  they  are  to  complete  the 
course  in  the  7  weeks  normally  allotted  for  the  program.  Gener¬ 
ally,  a  mastery  approach  to  the  course,  where  complete  facility 
with  the  material  attained,  typical  in  the  first  days,  must  be 
abandoned  by  all  but  those  officers  with  a  substantial  background 
in  German.  Most  students  have  to  develop  new  learning  strategies 
and  priorities  since  language  studies  are  quite  different  than 
other  courses  of  study  to  which  students  have  been  exposed.  This 
novel  learning  experience  can  be  quite  threatening  and  frustrat¬ 
ing  to  persons  used  to  career  and  academic  success.  The  instruc¬ 
tors  thus  become  counselors  and  diplomats  as  well  as  teachers,  a 
fact  which  may  enhance  the  reported  contribution  of  instructors 
to  the  course. 

Before  beginning  the  German  Gateway  course  at  DLI,  students 
are  supposed  to  receive  and  complete  materials  constituting  the 
"Headstart  Program"  (tapes,  texts  and  workbooks)  at  their  pre¬ 
vious  duty  assignment.  In  fact,  of  the  few  who  receive  it,  fewer 
complete  the  material.  The  years  of  experience  by  the  Gateway 
staff  with  this  situation  have  led  to  the  practice  of  allocating 
1  to  2  weeks  at  the  beginning  of  the  Gateway  program  for  coverage 
of  the  Headstart  material. 

The  pre-IVD  Headstart/Gateway  course  is  generally  regarded 
at  DLI  as  excellent,  though  "traditional"  in  nature.  That  is, 
the  Headstart  materials  are  pure  memorization  and  the  materials 
are  "grammar  driven"  rather  than  having  the  communicative  peda¬ 
gogical  basis  to  language  teaching.  Interest  in  IVD  materials 
is,  in  part,  related  to  the  German  Gateway  staff's  desire  to 
supplement  the  existing  course  with  more  communicative  material. 

The  materials  and  the  instructors  for  the  German  Gateway 
Course  need  to  be  adaptable  to  a  broad  range  of  language  experi¬ 
ence  and  ability.  The  population  of  the  program  consists  of 
mature,  career-established  officers,  in  contrast  to  the  typical 
DLI  initial  entry  students.  There  is  no  language  aptitude  re¬ 
quirement  for  the  course,  so  there  is  not  only  a  considerable 
variability  in  general  language  experience  and  German  language 
and  cultural  exposure,  but  also  in  language  aptitude.  Gateway 
students  may  have  had  no  prior  language  training  whatsoever  or 
may  have  had  years  of  training  in  several  languages.  Some  stu¬ 
dents  have  had  several  years  of  German  and  spent  extensive  time 
in  Germany,  while  others  have  had  neither  German  training  nor 
time  in  Germany. 

Dependents  are  permitted  to  participate  in  the  course  on  a 
space-available  basis.  Few  dependents,  however,  complete  the 
full  program. 

New  students  enter  the  Gateway  program  approximately  every  2 
weeks  so  that,  at  any  given  time,  the  students  in  the  program 
will  be  at  different  points  in  the  course.  The  size  of  the  Ger¬ 
man  Gateway  student  population  varies  on  an  annual  basis,  with 
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peak  periods  occurring  from  January/February  through  April/May  of 
each  year  when  the  total  number  in  the  program  may  reach  as  many 
as  fifty.  For  the  rest  of  the  year,  the  Gateway  population  may 
vary  from  as  few  as  three  to  five  in  the  whole  program  to  about 
ten  to  fifteen. 

Just  as  the  size  of  the  student  population  varies,  so  does 
the  number  of  instructors  involved  with  the  program.  During  tJ  i 
time  of  peak  student  population  there  may  be  as  many  as  ten  in¬ 
structors,  or  as  few  as  one  during  low  enrollment  periods.  The 
policy  is  to  assign  instructors  to  the  program  as  needed  to  main¬ 
tain  a  student  to  instructor  ratio  of  five  to  one.  However, 
depending  on  other  course  demands  on  the  German  Department  staff, 
and  the  overlap  of  in-coming  and  exiting  students  in  the  program, 
the  ratio  at  times  reaches  seven  to  one.  Instructors  are  rotated 
through  the  Gateway  course  for  six  month  periods  except  when 
temporarily  augmenting  the  staff  during  peak  period  coverage. 

Instructors  generally  regard  teaching  in  Gateway  as  a  de¬ 
sirable  assignment.  The  teaching  is  a  welcome  change  of  pace 
from  the  classroom  routine  since  it  affords  the  opportunity  to 
work  individually  with  students.  Of  course,  despite  official 
policy,  instructors  bring  with  them  their  own  methods  and  orien¬ 
tations  to  language  teaching.  They  may  be  "traditional,"  "com¬ 
municative,"  or,  most  likely,  have  their  own  individual  approach 
incorporating  elements  they  are  comfortable  with  from  a  variety 
of  methods.  Their  orientation  does,  however,  affect  how  recep¬ 
tive  they  are  to  a  communicatively  oriented  technological 
innovation. 

Objectives 

In  the  design  of  this  research  investigation,  three  objec¬ 
tives  were  identified.  The  first  objective  was  to  compare  the 
effectiveness  and  acceptability  of  the  Video  Enhanced  German 
Gateway  program  with  the  existing  German  Gateway  program.  The 
second  objective  was  to  identify  user  issues  in  the  adoption  of 
the  interactive  video  course  materials.  The  third  objective  was 
to  describe  issues,  factors,  and  conditions  in  the  planning  and 
development  of  the  IVD  materials  which  facilitated  or  impeded 
project  development.  This  report  presents  findings  supporting 
the  first  and  second  objectives.  Findings  relevant  to  the  third 
objective  are  planned  for  a  subsequent  report. 

METHOD 


Subjects 

The  sample  for  this  research  consisted  of  89  participants  in 
the  German  Gateway  program  at  DLI.  Ten  (11%)  of  the  sample  were 
dependents  of  military  personnel  in  the  program.  The  control  or 
Baseline  group  (N=49)  of  subjects  began  the  course  between  July 
1984  and  April  '985  before  the  interactive  video  disk  materials 
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for  the  German  Gateway  program  were  developed.  The  Interactive 
videodisc  (IVD)  Field  Test  group  (N=40)  started  the  course 
between  March  and  July  1986. 

Instruments 


The  German  Gateway  Pre-Course  Questionnaire  (Appendix  A)  wa3 
designed  to  provide  information  regarding  previous  language 
training,  students'  expectations  in  terms  of  German  functional 
ability  as  a  result  of  taking  the  course,  and  general  attitudes 
about  the  importance  of  the  course.  Pilot  testing  of  the 
questionnaire  revealed  a  strong  concern  of  the  subjects  about  the 
potential  for  identifying  them  with  their  responses.  Therefore, 
only  minimal  personal  information  was  requested  and  an  attempt 
was  made  to  minimize  their  concern.  For  example,  age  category 
was  substituted  for  rank,  as  providing  roughly  equivalent 
information. 

The  Entrance  Proficiency  Examination  (Appendix  B)  was 
developed  specifically  for  this  research  by  DLI  to  measure  the 
degree  of  functional  German  language  ability  possessed  by 
entering  participants.  The  test  measures  skills  from  simple 
vocabulary  recall  based  on  German  language  exposure  or  prior 
course  work  to  minimal  functional  German  language  capacity.  The 
examination  was  intended  to  provide  the  research  with  the  primary 
measure  for  assessing  group  comparability  of  initial  German 
language  skill. 

The  Post-Course  Questionnaire  (Appendix  C)  was  developed  to 
assess  students'  perceptions  of  the  effectiveness  of  the  course 
along  several  dimensions,  the  importance  of  various  components  of 
the  course,  and  their  satisfaction  with  their  language  skill 
achievement . 

An  IVD  use  Questionnaire  (Appendix  D)  was  developed  to  assess  the 
attitudes  of  users  toward  the  IVD  materials  and  its 
capabilities.  This  instrument  was  administered  only  to  students 
in  the  IVD  Field  Test  group.  The  primary  measure  used  to  assess 
end-of-course  group  differences  in  learning  German  was  an  Oral 
Proficiency  Examination  (OPE)  (Appendix  E)  that  was  developed  by 
the  project  Subject  Matter  Expert  specifically  for  the 
evaluation.  The  exam  consisted  of  nine  questions  with  a  total 
score  of  460  points  based  on  scores  for  pronunciation  (45  pts.), 
fluency  (40  pts.),  socio-cultural  linguistic  ability  (15  pts.), 
grammar  (95  pts.),  vocabulary  (140  pts.),  and  tasking  (125  pts.). 

As  a  part  of  the  research,  German  Gateway  instructors  wrote 
student  summaries  at  the  end  of  the  course  for  each  student.  The 
purpose  was  to  identify  any  unique  circumstances  or 
characteristics  of  individual  students  which  the  instructor 
thought  relevant  to  their  learning. 
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In  general,  the  Instructor  summaries  served  to  substantiate 
other  data;  for  example,  that  the  person  had  a  good  language 
background  and  worked  conscientiously,  or  had  a  rough  time  and 
barely  made  It  through  the  course.  The  only  Instance  In  which  an 
Instructor  summary  Identified  significant  factors  that  were  taken 
Into  account  In  the  data  analysis  was  In  providing  the  background 
for  a  single  extreme  case.  In  the  preliminary  analysis  of  the 
baseline  data.  It  was  noted  that  several  factors  combined  to 
create  an  "exceptional  case"  that  should  be  matched  In  the  field 
test  or  dropped  from  the  comparative  analysis.  This  person  was 
able  to  complete  only  six  modules,  had  an  extremely  low  score  on 
the  OPE  and  received  a  certificate  of  attendance  rather  than  the 
diploma.  Since  this  situation  Is  rare  but  does  occur  In  the 
program,  the  subject  was  retained  as  part  of  the  baseline  sample 
pending  determination  of  a  match  In  the  field  test.  The  failure 
to  Identify  a  comparable  situation  in  the  field  test  group 
resulted  in  the  case  being  dropped  from  the  comparative  analysis. 

German  Gateway  IVD  System 

The  hardware  for  the  German  Gateway  IVD  System  is  nine  units, 
each  consisting  of  a  videodisc  player,  color  monitor,  headset, 
and  controlling  micro-computer  with  dual  3  1/2  Inch  disk 
drives.  The  courseware,  produced  by  Brigham  Young  University 
under  contract  to  DLI,  Is  a  series  of  lessons  based  on  everyday 
situations.  The  video  portion  was  filmed  in  Germany.  Each 
lesson  is  on  Its  own  3-1/2  inch  disk,  while  the  video  and  audio 
for  several  lessons  are  on  a  single  platter.  Lessons  deal  with 
such  situations  as;  telephone,  train,  post  office,  shopping, 
apartment  renting  and  restaurant.  The  system  supports  several 
special  features  in  addition  to  playing  and  reviewing  the 
situation.  These  features  include  keyword/phrase  help  screens, 
text  with  audio,  vocabulary  help  screens,  grammar/cultural  notes, 
questions  and  help  screens  while  working  with  the  questions. 

Procedure 


Subjects  completed  both  the  German  Gateway  Pre-course 
Questionnaire  and  The  Entrance  Proficiency  Examination  at  the 
beginning  of  the  course.  At  the  completion  of  the  course  they 
completed  the  Post  Course  Questionnaire  and  were  administered  the 
Oral  Proficiency  Examination.  The  Instructor  Summaries  and 
Individual  course  module  scores  were  also  obtained.  Only  the  IVD 
group  completed  the  IVD  use  questionnaire. 

Group  comparability 

Demographics .  No  statistically  significant  differences 
between  Baseline  and  Interactive  Video  Disk  (IVD)  Field  Test 
groups  of  German  Gateway  students  were  found  for  age  (M^O), 
education  ( 9 completed  college,  77%  had  advanced  degrees),  or 
experience  In  Germany  (9^!&  had  been  to  Germany). 
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Table  1  shows  the  length  of  tine  spent  in  Germany  for  both 
groups.  The  Baseline  group  reflected  the  common  three  year 
assignment  as  the  modal  pattern,  however  the  IVD  shows  a  bimodal 
pattern,  with  many  of  the  officers  having  two  Germany  tours. 


Table  1 

Length  of  Time  Spent  in  Germany 
by  Group. 


Length  cf  Time 

B 

IVD 

N 

% 

N 

% 

Less  than  1  week 

2 

4 

2 

5 

Less  than  1  Months 

0 

0 

1 

3 

Less  than  1  Year 

1 

2 

2 

5 

1-2  Years 

2 

4 

3 

8 

2-4  Years 

29 

60 

13 

33 

4-6  Years 

8 

17 

5 

13 

Over  6  Years 

2 

4 

12 

30 

Total 

44 

38 

Using  actual  months  in  Germany,  a  significant  difference, 
(£(87)  ■»  2.02,  e  «  <• 05)  was  found,  the  IVD  group  having  spent 
more  time  in  Germany.  This  finding  suggests  the  need  to  consider 
the  "time  in  Germany"  variable  in  the  analysis  of  performance 
data.  There  is  no  obvious  explanation  for  this  difference.  Of 
course,  the  two  groups  selected  were  almost  a  year  apart,  and 
there  were  some  differences  in  the  program  cycles  from  which  the 
samples  were  drawn.  The  Baseline  group  was  about  equally  split 
between  peak  and  slack  inputs  while  the  Field  Test  group  was 
drawn  largely  from  the  peak  input  period.  There  were  no  known 
differences  or  changes  in  personnel  selection  or  assignment 
processes  within  the  military  related  to  this  sample  difference. 

Language  training.  Forty-eight  percent  of  the  students  in 
both  groups  had  had  some  German  language  study  with  no 
significant  group  difference.  Duration  of  German  study  ranged 
from  less  than  one  semester  in  high  school  to  36  total  semester 
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hours.  The  conditions  of  the  German  training  (educational  level, 
type  of  system,  and  numbers  of  years  since  trained)  did  not 
differ  significantly  between  the  two  groups. 

Sixty-seven  percent  of  the  subjects  had  had  previous  training 
in  other  foreign  languages.  No  statistically  significant 
difference  was  found  between  the  groups.  Of  those  with  previous 
other  foreign  language  study,  the  median  reported  number  of 
semester  hours  of  study  was  10.8.  Group  differences  were  non¬ 
significant  . 

The  Entrance  Proficiency  Examination  (EPE)  constituted  the 
primary  measure  for  assessing  group  comparability  of  initial 
German  language  skill.  Table  2  presents  the  results  for  the  two 
groups . 


Table  2 

German  EPE  Scores  by  Group. 


EPE  Total  Score 

Base 

IVD 

(300  possible) 

N 

% 

N 

* 

0 

11 

22 

12 

30 

1-10 

7 

14 

2 

5 

11-50 

12 

15 

8 

20 

51-100 

6 

12 

8 

20 

101-150 

6 

12 

3 

8 

151-200 

6 

12 

5 

13 

201-250 

1 

2 

2 

5 

M  -  55.92 
SIT  -  63.73 

M  -  64.28 
Sl>  -  71.68 

The  difference  between  the  groups  is  not  statistically 
significant  (_t_( 87)  ■  .58,  >  .5),  indicating  the  baseline  and 

IVD  groups  do  not  differ  in  initial  German  language  ability. 

Attitude  toward  learning  German.  On  the  Pre-course 
Questionnaire  subjects  responded  to  a  series  of  statements  about 
the  possible  relevance  of  and  reasons  for  learning  German. 
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Table  3  shows  these  data.  The  two  groups  showed  no  significant 
differences  in  their  responses  to  these  items  so  the  data  have 
been  combined.  Most  dependents  did  not  answer  the  duty-related 
items.  Responses  indicate  that  the  importance  of  the  course  is 
more  related  to  the  ability  to  function  in  Germany,  either  on  or 
off  the  Job,  than  strictly  to  career  interests. 


Table  3 


Response  Distribution  for  Pre 

-Course 

Attitude 

Statements 

• 

strongly 

agree 

agree 

disagree 

strongly 

disagree 

1.  Knowing  German  will  be 

important  in  performing  my 

official  duties  in  Germany. 

50 

35 

2 

0 

2.  Taking  this  course  is 

relevant  to  ray  army  career. 

26 

40 

14 

3 

3.  Understanding  German 

culture  will  be  important 

to  me  while  in  Germany. 

69 

20 

1 

0 

4.  I  need  to  learn  German 

in  order  to  function  in 

German  society. 

48 

37 

3 

0 

5.  I  do  not  expect  to  have 

any  trouble  Learning  German. 

7 

39 

37 

5 

Functional  language  expectations.  On  both  the  pre  and  post- 
course  questionnalresV  subjects  were  asked  to  respond  to  a  10- 
ltera  list  of  specific  foreign  language  use  situations  in  terms  of 
skills  they  expected  at  the  completion  of  Gateway.  The  items 
were  taken  from  a  survey  (Ryan-Jones  &  Burns,  1981)  where  Army 
commanders  in  Germany  identified  the  kinds  of  activities  for 
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which  they  needed  German  language  skill.  Only  the  pre-course 
responses  are  presented  here  in  Table  4  to  compare  the  initial 
expectations  of  the  Baseline  and  IVD  groups.  The  post-course 
responses  will  be  presented  later. 

To  make  the  pattern  clearer,  the  "very  easily"  and  "easily" 
responses  have  been  combined  into  a  "can  do"  category.  "With 
difficulty"  and  "with  great  difficulty"  are  combined  into  a 
"cannot  do"  category.  Data  are  shown  as  percentages  for  ease  of 
depicting  comparisons.  Some  dependents  did  not  respond  to  the 
duty  related  items,  accounting  for  totals  of  less  than  100?  (See 
Appendix  A  for  Questionnaire  and  complete  item  wording.) 


Table  4 


Percentages  by  Group  of 
Language  Activities. 

Ability 

Expectations 

for  German 

Activities 

Can 

Do 

Cannot 

Do 

B 

IVD 

B 

IVD 

Hail  Taxi 

100 

98 

0 

3 

Small  Talk 

57 

53 

43 

48 

Hotel  Arrangements 

98 

98 

2 

3 

Radio  Weather 

65 

50 

35 

50 

Deal  with  Police 

54 

48 

46 

53 

Military  Personnel 

57 

63 

41 

38 

Official  Correspondence 

17 

23 

83 

78 

Make  a  Speech 

38 

43 

63 

58 

Official  Phone  Call 

31 

40 

69 

60 

Meet  Gov't  Official 

50 

43 

50 

58 

Both  groups  show  a  similar  pattern  of  being  somewhat  more 
conservative  in  their  estimates  of  duty  related  abilities.  The 
differences  between  the  Baseline  and  IVD  groups  are  not 
statistically  significant  for  any  of  these  items. 
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RESULTS:  PROFICIENCY  AND  ATTITUDES 
IVD  Impact  on  German  proficiency 

The  Oral  Proficiency  Examination  (OPE)  was  administered  to 
each  student  by  two  DLI  trained  and  certified  testers.  Ideally, 
the  testing  would  have  been  done  by  a  team  of  qualified 
proficiency  testers  not  Involved  in  the  project.  However, 
practical  considerations  lead  to  testing  being  conducted  by  pairs 
of  available  test-qualified  instructors  teaching  in  German 
Gateway  at  the  time,  including  the  project  Subject  Matter 
Expert.  The  scores  of  the  two  raters  were  averaged  in  arriving 
at  a  final  student  score. 

Seven  raters  were  involved  in  the  Baseline  group  testing  with 
four  persons,  in  varying  combinations,  administering  most  of  the 
tests.  For  the  IVD  Field  Test  group,  33  of  40  tests  were 
administered  by  the  same  two  testers  with  one  of  the  two  testers 
involved  In  each  of  the  other  seven  tests.  This  pair  of  testers 
Included  the  Subject  Matter  Expert  and  the  one  test-qualified 
instructor  (not  a  Baseline  group  tester)  in  the  program  at  the 
time  of  the  field  test. 

One  rater  in  this  same  pair  of  raters  had  consistently  and 
significantly  higher  ratings(M  -  326.94)  than  the  other  rater  (M 
«  289.73),  _t(32)  *  6.85,  £7  .001.  This  disparity  was  reduceT 

by  dropping  one  dimension  on  the  test  where  the  discrepancy  was 
greatest  and  reflected  a  different  rating  criterion.  When  this 
wis  done,  the  statistically  significant  difference  between  the 
two  raters  was  eliminated  t^(32)  -  1.16,  j).  >  .2. 

Analysis  of  covariance  performed  on  the  total  remaining 
points,  with  months  in  Germany  as  the  covariant,  indicated  a 
difference  between  the  two  groups  with  the  Field  Test  group  (M^  ■ 
214.04)  performing  significantly  better  than  the  Baseline  group 
( M^  *  155*78),  t_(85)  *  5.54,  <  .001.  This  finding  suggests  the 

students  using  the  IVD  materials  achieved  greater  proficiency 
than  those  in  the  Baseline  group.  However,  because  of  the 
questionable  tenability  of  any  strict  assumption  of  randomness  of 
rater  assignment  for  scoring  of  the  the  OPE,  we  examined  another 
variable  related  to  the  OPE  but  less  affected  by  rater  Judgments. 

Examination  of  the  OPE  protocols  revealed  much  of  the  group 
difference  in  OPE  scores  is  attritutable  to  a  difference  in  the 
number  of  questions  students  attempted.  To  clarify  this 
difference,  we  calculated  the  number  of  questions  each  student 
missed.  We  employed  a  conservative  coding  scheme.  For  the 
Baseline  group.  If  either  rater  gave  any  points  the  question  was 
counted  as  answered.  For  the  IVD  Field  Test  group,  a  question 
was  counted  as  a  missed  question  if  either  rater  gave  zero  points 
for  the  question.  Even  with  this  approach,  the  difference 
between  the  groups  i3  apparent  in  Table  5. 
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Table  5 

Missed  OPE  Questions  by  Group 


Base 

IVD 

Number 

N 

% 

N 

% 

0 

10 

21 

30 

75 

1 

1M 

29 

7 

18 

2 

15 

31 

3 

8 

3 

9 

19 

0 

00 

While  both  the  combined  raters  scores  on  the  OPE  and  the 
Items  missed  on  the  OPE  tend  to  support  the  effectiveness  of  the 
IVD,  unfortunately  these  data  cannot  be  Interpreted  as 
necessarily  supporting  the  positive  effects  of  the  IVD.  The 
findings  are  complicated  by  the  fact  that  the  program  began  using 
a  new  German  Gateway  Pinal  Examination  (GGPE)  at  the  same  time 
the  IVD  field  test  began.  The  decision  of  the  German  Department 
to  change  the  GGPE  was  Intended  to  promote  a  proficiency-oriented 
approach.  As  discussed  above,  the  use  of  IVD  was  also  Intended 
to  support  the  proficiency  orientation.  Prom  a  programmatic 
point  of  view  It  made  very  good  sense  to  Initiate  both  changes 
simultaneously  as  part  of  a  communicative  overhaul  of  the 
program.  However,  the  fact  that  tests  tend  to  structure  teaching 
as  well  as  student  study  also  clouds  the  findings  of  this 
research. 

Problems  with  the  original  GGPE  resulted  In  Its  rejection  as 
an  adequate  performance  measure  for  the  IVD  evaluation.  Its 
discrimination  among  students  was  minimal;  it  had  been  used  for 
many  years  and  was  probably  compromised;  and  it  was  essentially 
an  achievement  test  measuring  specific  course  content  rather  than 
a  broader  concept  of  functional  ability  In  the  language.  When 
the  new  test  was  developed,  the  German  department  eagerly  and 
quickly  adopted  it.  The  new  test,  with  alternate  versions  of 
test  items.  Involved  role-play  situations  which  tested  the 
student' 8  ability  to  perform  In  the  language.  Table  6  shows  that 
the  new  GGPE  does  seem  to  have  greater  score  dispersion,  but  any 
further  comparison  between  the  groups  is  not  warranted. 

The  new  GGPE  Is  similar  but  not  identical  to  the  OPE  devised 
for  the  evaluation.  The  new  GGPE  scores  of  the  IVD  group  have  a 
slgnicantly  higher  correlation  with  the  OPE  scores  (r_  *  .81)  than 
the  original  GGPE  scores  of  the  Baseline  group  have  with  their 
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OPE  scores  (_r  =  .58),  ( Z_  -  2.00,  p  <  .05),  suggesting  the  new 
GGPE  and  the  OPE  are  more  closely  measuring  the  same  construct. 
Not  realizing  the  research  need  for  comparability  of  course 
tests,  the  German  Gateway  staff  was  pleased  with  what  they 
perceived  to  be  an  improved  test  for  student  evaluation 
purposes.  Although  ARI  explained  the  requirements  for  exact 
course  test  comparability,  in  the  final  decision  the  interests  of 
the  program  prevailed  over  the  requirements  of  the  evaluation  and 
the  new  test  was  adopted. 

As  a  result  of  the  course  testing  difference,  the  research 
findings  could  as  readily  be  considered  a  study  of  the  effects  of 
modifying  the  test  process  in  the  program  as  of  the  effects  of 
the  introduction  the  IVD.  The  conclusion  from  the  OPE  data  then 
must  be  that  there  is  evidence  of  improvement  in  student 
performance  that  could  be  attributable  to  either  the  IVD,  the  new 
course  test  structure,  or  to  a  combination  of  the  two  factors, 
both  contributing  to  proficiency-oriented  instruction. 


Table  6 

Score  Distribution  for  Baseline  on  Old  and  for  IVD  on  New  GGPE. 


GGPE  score 

Base 

IVD 

N 

% 

N 

% 

70-79 

6 

13 

9 

26 

80-89 

8 

17 

10 

29 

90-99 

32 

70 

15 

42 

100 

0 

0 

1 

3 

Total* 

46 

35 

•Missing  scores  are  mostly  dependents  who  usually  did  not 
take  the  final  exam. 


Another  performance  measure,  though  not  a  measure  of  tested 
proficiency,  is  the  number  of  instruction  modules  of  the  German 
Gateway  material  students  completed.  The  IVD  group,  generally, 
did  not  complete  as  many  modules  as  the  Baseline  group.  Table  7 
shows  these  data.  With  the  addition  of  the  IVD  material, 
apparently  the  total  time  required  to  complete  the  modules  is 
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longer,  and  does  not  allow  for  Including  the  five  supplemental 
modules.  In  addition,  some  students  tried  to  use  any  remaining 
time  after  completion  of  the  basic  11  modules  to  finish  or  review 
IVD  material.  Thus,  the  Inclusion  of  IVD  material  does  not  seem 
to  encourage  study  or  completion  of  the  full  16  modules  In  the 
German  Gateway  course,  and  may  Inhibit  it.  However,  the 
completion  of  fewer  modules  by  the  IVD  group  does  not  appear  to 
have  a  deleterious  effect  on  their  proficiency,  as  indicated  by 
the  measures  discussed  above. 


Table  7 


Modules 

Completed 

by 

Group.* 

Number 

Base 

IVD 

N 

% 

N 

% 

10 

0 

0 

3 

8 

11 

29 

62 

23 

58 

12 

0 

0 

6 

15 

13 

3 

6 

2 

5 

14 

0 

0 

0 

0 

15 

2 

4 

1 

3 

16 

13 

28 

5 

13 

»  Dependents  were  handled  differently  in  program.  Information 
not  obtained  for  baseline  dependents. 


IVD  Impact  on  student  course  assessment 


This  section  reports  the  results  from  the  Post-Course 
Questionnaire  administered  to  both  groups.  In  the  tables  which 
follow,  the  means  were  calculated  from  the  full  five  point 
scales.  However,  displayed  responses  are  collapsed  Into  the 
three  categories  for  simplification:  negative,  neutral,  and 
positive.  The  Baseline  N  is  45  and  the  IVD  N  is  36.  Differences 
between  baseline  and  IVD  were  tested  for  each  item  and  were 
statistically  significant  where  indicated. 


Table  8 


Percentages  by  Group  for  Effectiveness  Assessment  of  Course. 


Ineffective 

Neutral 

Effective 

Means 

Evaluation  Item 

B* 

V* 

BX 

V* 

BX 

vx 

Base 

IVD 

Pronunciation 

4 

6 

22 

25 

74 

69 

3.9 

3.9 

Grammar/Syntax 

20 

8 

33 

36 

48 

56 

3.4 

3.6 

Vocabulary 

4 

3 

28 

25 

67 

72 

3.8 

3.9 

Idioms 

15 

11 

39 

53 

46 

36 

3.4 

3.3 

The  IVD  did  not  result  in  significant  overall  increases  in 
the  students*  assessments  of  the  program  (Table  9).  Table  9 
shows  how  satisfied  students  were  with  their  German  language 
skills.  The  IVD  group  was  significantly  more  satisfied  with 
their  listening  comprehension  skill  than  was  the  Baseline 
group.  As  expected,  the  Field  Test  group  did  indicate  (Table  10) 
an  importance  of  the  audio-visual  material  comparable  to  that  of 
other  aspects  of  the  course,  and  this  rating  was  significantly 
higher  than  thac  of  the  Baseline  group  (which  had  available  only 
Headstart  videos  and  other  taped  material). 


Table  9 

Percentages  by  Group  for  Skill  Satisfaction. 


Dissatisfied 

Neutral 

Satisfied 

Means 

Evaluation  Item 

BX 

vx 

B% 

VX 

BX 

VX 

Base 

IVD 

Listening 

Comprehension 

24 

3 

35 

42 

41 

56 

3.2 

3.8» 

Speaking 

30 

3 

22 

61 

48 

36 

3.2 

3-4 

Reading 

13 

0 

13 

36 

74 

64 

3.8 

3.8 

*  t_(80)  =  2.98,  2.  <  *01 
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Table  10 


Percentages  by  Group  for  Importance  of  Course  Components. 


Unimportant 

Neutral 

Important 

Means 

Evaluation  Item 

Bit 

IVD*  Bit 

IVDJt 

Bit 

IVDJt 

Base  IVD 

Tapes 

7 

6 

33 

25 

61 

69 

3-7  3-9 

Texts 

7 

0 

28 

17 

65 

83 

3.9  4.25 

AV  Material 

61 

17 

17 

14 

22 

69 

2.3  3.7* 

Instructors 

0 

0 

13 

8 

87 

92 

4.6  4.7 

Quizzes/tests 

20 

14 

22 

28 

59 

58 

3.5  3.5 

Exercises 

2 

8 

28 

22 

70 

69 

4.0  3.8 

*N(B)  -  41  N( V) 

“  36 

1(75)  4 

.68  £  < 

.001 

Table  11 

Percentages  by  Group 

for  Adequacy 

of  Learning 

In  Specialized  Areas. 

Inadequate 

Neutral 

Adequate 

Means 

Evaluation  Item 

Bit 

n 

Bit 

n 

Bit 

V* 

Base  IVD 

Simulated  Real 
Life: 

Communication 

11 

3 

20 

19 

69 

78 

3.8  4.1 

Cultural  Info. 

16 

3 

18 

34 

67 

63 

3.6  3.9 

16 


Table  12 


Percentages  by  Group  for 
Areas . 

Ea3ie3t 

and  Most 

Difficult  Learning 

Learning 

Easiest 

Most  Difficult 

BJ 

V* 

B? 

n 

Listening 

Comprehension 

9 

31  »« 

24 

6  « 

Pronunciation 

27 

14 

0 

3 

Grammar/Syntax 

7 

0 

40 

49 

Vocabulary 

9 

3 

2 

3 

Reading  Comp. 

49 

53 

2 

3 

Fluency 

0 

0 

31 

38 

N=45  (B)  35  (V) 
*  £_  <  .05 
**  £_  <  *01 


Tables  12  and  13  present  responses  to  questions  identifying 
language  areas  students  found  to  be  the  easiest  and  most 
difficult  to  learn  in  the  course.  Each  pair  of  proportions  in 
Table  13  was  tested  for  differences  with  the  results  shown.  The 
difference  between  the  groups  in  listening  comprehension  is 
obvious.  Not  only  do  the  percentages  related  to  listening 
comprehension  vary  between  the  two  groups,  they  actually  reverse- 
-listening  comprehension  is  one  of  the  skills  regarded  as  easiest 
by  the  Field  Test  group  and  most  difficult  by  the  Baseline 
group.  Since  listening  comprehension  is  an  important  aspect  of 
the  communicative  approach  and  one  of  the  skills  the  IVD  directly 
supports,  these  data  suggest  that  the  IVD  may  have  contributed  to 
acquiring  listening  skills. 

Table  13  shows  the  resources  found  to  be  must  useful  keyed  to 
the  student's  identified  easiest  and  most  difficult  language 
areas.  Tests  of  the  differences  in  each  pair  of  percentages  were 
conducted  with  the  results  shown.  Students  in  the  IVD  group 
reported  that  audio  video  materials  contributed  to  their 
Identified  easiest  program  component  significantly  more  than 
students  in  Baseline  group  reported. 
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Table  13 

Percentages  by  Group  for  Most  Useful/Helpful  Program  Component. 


Easiest 

Most 

Difficult 

Component 

B? 

IVD* 

B? 

IVD? 

Tapes 

16 

11 

2 

3 

Texts 

49 

56 

16 

28 

Instructors 

34 

22 

80 

67 

Tests 

2 

0 

2 

3 

AV  Material 

0 

11  * 

0 

0 

•  <  .05 

Most  students  In  both  groups  reported  that  their  Interest 
either  Increased  (59?)  or  started  high  and  remained  so  ( 32% ) . 

The  primary  reason  given  for  the  increased  interest  was  the 
challenge  experienced  as  a  result  of  minimally  mastering  the 
basics  of  communicating  in  German. 

In  the  responses  to  the  item  about  appropriate  course  length 
only  three  people  in  the  Baseline  group  and  two  in  the  Field  Test 
group  thought  the  course  was  too  long,  all  having  extensive 
previous  training  in  German.  Half  in  each  group  (B»53?»  IVD  * 
56?)  found  the  current  6  to  7  week  length  of  the  course  to  be 
about  right,  with  most  of  the  remainder  recommending  an  8  to  12 
week  course.  The  introduction  of  the  IVD  did  not  result  in  any 
significant  differences  in  student  perceptions  of  the  appropriate 
course  length. 

The  open-ended  comments  made  by  the  Field  Test  group  focused 
on  issues  related  to  the  IVD  to  such  an  extent  that  the  same  sets 
of  categories  could  not  be  used  for  the  two  groups.  Only  two 
categories  were  relevant  to  both  groups.  Sixteen  people  in  the 
Baseline  and  nine  in  the  IVD  group  made  comments  related  to  the 
need  for  more  conversational  practice,  either  by  having  more 
Instructor  time  or  through  instructor  mediated  group  sessions. 
Four  Baseline  and  six  Field  Test  students  mentioned  the  need  for 
a  longer  course  in  their  comments  as  well  as  in  the  response  to 
the  appropriate  course  length  item.  Otherwise,  the  Baseline 
student  comments  dealt  with  the  need  for  a  review  of  English 
grammar,  the  need  for  exposure  to  a  greater  variety  of  German 
speakers,  and  more  military  or  professional  terminology.  The  IVD 
group  emphasized  the  need  to  continue  the  Headstart  part  of  the 
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course  ( 32% )  and  to  allocate  time  for  IVD  use  (18%).  Other 
comments  made  by  two  or  more  Individuals  Included  the  need  to 
better  Integrate  IVD  materials  Into  the  program,  for  more 
Instructor  encouragement  to  use  IVD,  and  to  have  the  facility 
available  at  night. 

IVD  Impact  on  self-assessed  German  skill 

At  the  end  of  the  course,  students  were  asked  to  assess  their 
abilities  to  perform,  in  German,  the  same  activities  which  they 
rated  prior  to  the  course  according  to  their  expectations.  Table 
l1!  shows  the  responses  compared  to  expectations  for  both 
groups.  The  responses  are  again  combined  into  dichotomous 
categories.  The  "can  do"  percentages  are  provided  along  with  the 
pre/post  differences  for  both  groups.  (See  Appendix  A  and  C  for 
complete  item  wordings).  None  of  the  between  group  differences 
are  statistically  significant,  nor  are  the  mean  pre/post  percent 
differences  for  combined  social  and  duty-related  activities  (M(B) 
=  -17X,  M_( V)  *  -15%).  However,  overall  it  seems  clear  that 
assessed  abilities  are  below  pre-course  expectations.  These 
values  probably  reflect  an  adjustment  to  somewhat  unrealistic 
initial  expectations  rather  than  having  much  to  do  with 
reflecting  course  satisfaction. 
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Table  14 

Pre/Post  Skill  Assessments  by  Group. 


"Can 

Do" 

Activities 

B% 

Pre 

V* 

B* 

Post 

V* 

Difference 

bz  n 

Hail  Taxi 

100 

98 

80 

89 

-20 

-9 

Small  Talk 

57 

53 

33 

36 

-24 

-17 

Hotel  Arrangements 

98 

98 

87 

92 

-11 

-6 

Radio  Weather 

65 

50 

33 

39 

-32 

-11 

Police  Situation 

54 

48 

28 

29 

-26 

-19 

N 

-49 

N=40 

N=46 

N=36 

Military  Personnel 

57 

63 

33 

17 

-24 

-46 

Of f .Correspondence 

17 

23 

20 

17 

+3 

-6 

Make  Speech 

38 

43 

36 

43 

-2 

0 

Off.  Phone  Call 

31 

40 

27 

29 

-4 

-11 

Meet  Gov't  Off. 

50 

43 

20 

17 

-30 

-26 

N 

=  48 

N=40 

N=45 

N=35 

IVD  Impact  on  Instructor  contact  time 

Does  Instructor  Involvement  with  the  IVD  Interfere  with  the 
time  available  for  students,  or  conversely,  do  students  use  the 
IVD  Instead  of  working  with  the  Instructors?  There  were  large 
differences  among  Instructors  in  the  amount  of  time  spent  with 
students.  An  Indication  of  this  variability  Is  evident  from  the 
mean  times  instructors  reported  spending  with  students.  For 
those  students  with  the  same  instructor  throughout,  the  means 
ranged  from  692  to  1705  total  minutes  (11.5  to  28.4  hours)  with 
instructors  for  the  Baseline  group,  and  798  to  1529  total  minutes 
(13*3  to  25.5  hours)  with  instructor  for  the  IVD  group.  Given 
this  extreme  variability  plus  the  fact  that  the  IVD  Field  Test 
group  has  greater  variance  in  times  (£(36,44)  *  1.86,  j>_  <  .05), 
the  significant  statistical  difference  between  the  groups 
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indicated  by  the  data  in  Table  15  is  probably  best  interpreted  as 
indicating  there  is  no  evidence  that  the  IVD  reduces  the  amount 
of  student/instructor  contact  time  and  may  increase  it.  The 
increase  could  be  accounted  for  by  increased  instructor  and/or 
student  enthusiasm  during  the  IVD  Field  Test  phase  of  the 
research  due  to  factors  contributing  to  the  Hawthorne  effect. 


Table  15 


Mean  Instructor-reported  Time 

Spent  with 

Students 

by  Group. 

B 

IVD 

Minutes 

N 

* 

N 

* 

200-399 

0 

0 

2 

5 

400-599 

4 

8 

1 

2.5 

600-799 

6 

12 

1 

2.5 

800-999 

20 

41 

1 

2.5 

1000-1199 

9 

18 

5 

12.5 

1200-1399 

1 

2 

8 

20 

1400-1599 

2 

4 

10 

25 

1600-1799 

2 

4 

5 

12.5 

1800-1999 

1 

2 

4 

10 

Total 

45 

91* 

37 

92.5* 

Baseline 

t4»968  (16  hours) 

SD-296 

Field  Test 

JM-1344  (22  hours) 

SD-404 

_t_( 80 )  »  4.92,  £_  < 

.001 

IVD  Impact  on  student  study  patterns 

Of  interest  in  evaluating  the  IVD  was  the  effect  the  IVD 
might  have  on  student  study  time.  As  an  additional  course 
requirement,  would  it  mean  more  time  involved  with  course 
material,  thereby  increasing  the  total  "time  on  task"?  Or,  would 


21 


it  enhance  the  learning  process  in  a  way  that  actually  reduced 
the  time  needed  to  learn  a  given  amount  of  material?  The 
original  intent  was  to  relate  study  time  to  modular  test 
scores.  However,  instructors  did  not  keep  sufficiently  complete 
records  of  modular  tests.  The  effort  to  collect  results  of 
modular  self-tests  was  so  unsatisfactory  for  the  Baseline  group, 
that  the  effort  was  dropped  for  the  Field  Test.  It  was  difficult 

to  get  students  to  remember  to  keep  an  on-going  record  of  their 

study  time,  so  this  information  is  missing  for  31?  of  the 
Baseline  group  and  40?  for  the  Field  Test  group.  What  data  were 
obtained  on  the  number  of  hours  studied  shows  a  significant 
difference  (£  <  .01),  with  the  Field  Test  group  reporting  more 
total  hours  of  study  (M^  ■  178)  than  the  baseline  group  (M^  ■  130) 
for  the  basic  11  modules.  By  Itself,  these  data  do  not  say  very 
much,  especially  with  the  high  proportion  of  missing  data.  .In 
the  context  of  information  presented  in  the  next  section, 
however,  it  may  be  more  meaningful.  Students  reported  they  often 

used  the  IVD  as  a  break  from  other  course  work.  Thus,  they  may 

have  continued  their  involvement  with  German  language  beyond  what 
might  otherwise  have  been  the  case. 

IVD  use  and  Impact 


In  this  area  there  are  responses  only  from  the  Field  Test 
group.  The  relevant  data  derive  from  the  IVD  questionnaire  (see 
Appendix  D),  station  problem  logs,  and  observation  and  interview 
notes.  Frequency  data  will  be  presented  from  the  IVD 
questionnaire,  and  any  numbers  related  to  comments  or  problems 
are  reported  if  made  by  more  than  one  person.  Otherwise, 
comments  and  suggestions  are  presented  as  Information  of  general 
interest  to  indicate  the  type  and  range  of  comments  made.  Some 
points  were  made  by  the  same  person  both  orally  in  discussions 
with  the  observer,  and  in  writing  on  the  questionnaire,  and  so 
are  not  additional  data.  Both  are  Included  because  a  broad  range 
of  points  was  made  In  the  interviews,  while  students  probably 
wrote  down  only  those  they  felt  most  strongly  about  or  thought 
were  most  important. 

Equipment  problems.  Problem  log  sheets  were  attached  to  each 
of  the  IVD  stations;  during  the  orientation  to  the  evaluation 
process,  students  were  requested  to  log  any  equipment-related 
difficulties  they  encountered.  Of  the  nine  stations,  only  three 
logs  sheets  contained  any  entries.  In  one  instance,  a  cable  had 
come  loose  and  was  quickly  fixed.  The  other  recorded  problems, 
and  several  unrecorded  but  noted  in  the  observations,  involved 
diskettes  which  did  not  function  properly.  The  students  quickly 
learned  to  simply  exchange  the  diskette  when  they  had  this 
problem. 

In  addition  to  the  fact  that  many  students  did  not  realize 
the  "Speech"  lesson  required  a  unique  second  computer  disk,  there 
seemed  to  be  bugs  in  the  programming  of  that  particular  lesson 
which  resulted  in  several  students  Just  giving  up  on  it.  As  of 
the  end  of  the  Field  Test  the  "Per  Diem"  game,  which  is  the  most 
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interactive  lesson  in  the  set,  was  still  not  in  regular  use.  A 
few  people  had  tried  it  out  and,  apparently,  ran  into  problems, 
indicating  the  need  for  further  debugging. 

Five  students  in  the  interviews  and  eight  on  the 
questionnaire  expressed  frustration  with  the  slowness  of  system 
response,  and  thought  that  more  rapid,  if  not  instantaneous, 
response  should  be  possible.  Three  people  commented  in  the 
interviews  on  the  ease  of  use  or  "user  friendliness"  of  the 
system,  and  92%  of  the  questionnaire  respondents  reported  that 
they  found  the  IVD  "easy"  or  "very  easy"  to  use.  Ninety-five 
percent  of  the  respondents  indicated  they  were  "usually"  or 
"always"  able  to  use  the  IVD  when  they  wanted,  although  two 
persons  did  suggest  that  the  facility  be  open  and  equipment  made 
available  at  night. 

Other  suggestions  relevant  to  the  equipment  were: 

"there  should  be  a  master  switch  to  turn  everything  on" 

"the  systems  should  be  turned  on  in  the  morning  and  left 
on  for  the  day" 

"make  it  easier  to  go  backwards  and  forwards  and  sign 
off" 

"put  the  stations  in  the  student  rooms,  away  from 
distractions"  (from  two  phase  II  persons;  see 
observation  section) 

In  general,  equipment  related  problems  did  not  seem  to  be  a 
major  factor  in  the  general  use  of  the  IVD,  or  in  individual 
decisions  about  using  the  system.  Rather,  specific  lessons, 
namely,  the  "Speech"  and  "Per  Diem"  lessons  were  little  used. 

Courseware  and  Software  problems.  The  major  complaint 
related  to  the  course  was  the  lack  of  a  glossary  or  some  form  of 
vocabulary  helps  in  English.  This  was  mentioned  by  all  but  a  few 
persons  in  the  interviews  and  written  in  as  a  recommendation  for 
improving  the  system  by  15  people  on  the  questionnaire,  more  than 
any  other  item.  Four  people  mentioned  that  some  lessons  were  too 
long  ("Just  go  on  and  on,  for  example,  'Tankstelle'") .  Three 
people  mentioned  that  the  initial  lessons  were  too  difficult,  a 
perception  generally  supported  by  the  staff.  Five  people  in  the 
interviews  and  three  on  the  questionnaire  wanted  greater 
interactivity  in  the  program.  Problems  with  the  sound  or  speech 
("too  fast")  on  specific  lessons  were  mentioned,  with  the 
"Vacation"  lesson  singled  out  specifically. 

Physical  problems.  On  the  questionnaire,  only  four  persons 
indicated  they  experienced  any  physical  problems  relating  to  IVD 
use.  Two  persons  requested  either  adjustable  chairs  or  stations, 
so  the  viewing  angle  could  be  adjusted  according  to  varying 
heights  of  viewers.  One  person  found  the  headset 


23 


uncomfortable.  And  one  Individual,  who  was  color-blind,  reported 
persistent  difficulty  with  the  extensive  use  of  the  color  red  In 
the  directions.  None  of  the  Individuals  Indicated  that  the 
problems  Inhibited  their  use  of  the  system,  but  were  discomforts 
that  showed  up  when  they  used  the  system  for  extended  lengths  of 
time. 

Procedure  problems.  The  major  Issue  concerning  procedures  Is 
related  to  the  reported  difficulty  with  the  first  lessons.  The 
students  not  only  had  to  try  to  get  something  out  of  the  lessons 
presented  In  German,  but  also  had  to  become  familiar  with  the  IVD 
Instructions  In  German,  all  at  once — a  lot  to  take  In  at  one  time 
and  quite  threatening,  even  with  the  help  feature  which  presented 
an  English  translation  of  the  instructions.  Very  few  people  were 
observed  or  mentioned  using  the  vocabulary  and  grammar  helps 
while  working  with  the  questions,  even  though  that  should  have 
been  the  context  in  which  these  helps  could  have  been  most 
meaningful.  The  students  generally  figured  out  a  minimal  set  of 
basic  instructions  that  would  allow  them  to  work  with  the  IVD  and 
continued  to  rely  on  that  basic  set  throughout  the  course. 
Typically,  this  basic  set  did  not  include  the  specialized 
instructions,  for  example,  how  to  fill  In  the  blanks  In  the 
questions  part  of  the  program. 

A  common  complaint  and  one  on  which  students  asked  for  help, 
was  how  to  exit  the  system  instantaneously.  They  only  knew  how 
to  exit  from  the  menu,  which  meant  if  they  were  at  the  tell 
(chapter)  level  they  had  to  go  down  to  the  sub-segment  level  to 
get  to  the  menu.  If  they  were  In  a  sub-segment  they  generally 
had  to  go  through  several  steps  to  get  back  to  the  main  menu  In 
order  to  exit. 

Some  students  had  difficulty  using  the  system  passively  If 
they  wanted.  That  is,  they  could  not  view  the  material  at  the 
tell  level  and  continue  on  through  the  next  tells,  without  going 
into  the  Interactive  segment  level. 

Training  on  IVD  equipment.  In  response  to  the  questionnaire 
Item  about  adequacy  or  tne  instruction  received  on  use  of  the 
equipment,  no  one  reported  that  it  was  Inadequate.  Responses 
began  at  "adequate"  (N*5),  and  increased  as  the  scale  moved  in 
the  positive  direction:  "quite  adequate"  (N-13),  and  "completely 
adequate"  (N*17).  Not  only  were  the  individual  students 
thoroughly  briefed  (45  minuties  to  an  hour  each)  on  the  use  of 
the  system,  but  the  availability  of  the  Subject  Matter  Expert  for 
the  system  throughout  the  Field  Test  enabled  students  to  get  any 
needed  assistance  as  they  got  into  actual  IVD  use  during  the 
course. 

In  addition  to  the  orientation  provided,  each  station  had  a 
written  introduction  to  and  overview  of  the  system.  With  the 
ready  availability  of  the  Subject  Matter  Expert  these  were  never 
observed  being  referred  to.  However,  under  normal  conditions, 
without  the  Subject  Matter  Expert  readily  at  hand,  the  written 
orientations  might  become  essential. 


IVD  features .  The  questionnaire  asked  respondents  to  assess 
the  importance  and  usefulness  of  various  general  IVD 
characteristics  and  specific  features  of  the  German  Gateway  IVD 
materials.  Tables  16  and  17  present  the  results. 


Table  16 

User  Assessed  Importance  of  IVD  Characteristics. 


Very  Unimportant 

1 

2 

3 

4 

5 

Very 

Important 

Immediate  Feedback 

2 

2 

4 

13 

14 

(M^  = 

4.00) 

Visual  Presentation 

2 

2 

2 

16 

13 

(M  - 

4.03) 

Variety  of  Speakers 

2 

2 

4 

10 

17 

(M  - 

4.09) 

Contextual  Learning 

3 

0 

3 

14 

14 

(M_  - 

4.06) 

Active  Involvement 

3 

2 

12 

12 

6 

(M_  - 

3.46) 

Interactivity 

2 

1 

11 

12 

9 

(M_  - 

3.71) 

Most  of  the  unique  characteristics  of  the  IVD  instructional 
mode  were  considered  important  by  the  Field  Test  students,  with  a 
noticeable  difference  between  the  last  two  items  (active 
involvement  and  interactivity).  This  may  be  due  to  students 
perceiving  the  German  Gateway  IVD  materials  as  not  particularly 
interactive. 
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Table  17 


User  Assessed  Utility  of 

German  Gateway 

IVD 

Features 

. 

Useless 

1 

2 

3 

4 

5 

Useful 

Selecting  Sub-segments 

3 

4 

8 

16 

3 

(M  - 

3.35) 

Replay  Video 

4 

9 

5 

10 

6 

(M  - 

3.15) 

Keyword/Phrase  Helps 

2 

9 

7 

6 

10 

(M  - 

3.38) 

Text  with  Audio 

2 

1 

0 

7 

24 

01  - 

4.47) 

Vocabulary  Helps 

2 

2 

2 

17 

12 

(fl  - 

4.00) 

Grammar/Cultural  Notes 

3 

4 

8 

9 

11 

(M_  = 

3.60) 

Questions 

4 

1 

9 

11 

9 

(M_  - 

3.59) 

Access  Help  in  Questions 

5 

7 

11 

10 

2 

(M_  - 

2.91) 

Clearly  the  moat  favored  feature  of  the  IVD  program  from  the 
students'  point  of  view  was  the  ability  to  read  the  text  of  the 
video  material  they  have  Just  seen  (text  with  audio).  This 
feature  Is  also  the  most  compatible  with  other  portions  of  the 
original  German  Gateway  course  which  uses  audio  tapes  with 
accompanying  text.  It  Is  the  familiar  learning  mode  In  the 
course,  and  this  feature  takes  the  new  and  makes  It  familiar. 

In  general,  these  responses  reflect  the  typical  way  the 
students  were  observed  to  use  the  system:  view  the  lesson;  go  to 
the  text/audlo,  possibly  using  the  vocabulary  and 
grammar/cultural  helps;  then  finish  the  lesson  by  going  over  the 
questions,  but  not  using  the  helps  In  the  process.  They  rarely 
used  the  replay  video  or  the  keyword/phrase  feature  of  the  IVD. 

In  some  lessons  the  questions  are  presented  via  audio  rather 
text.  When  asked  about  their  preference  for  audio  or  text 
exercises,  the  majority  (N-20)  of  the  students  checked  the 
statement  that  they  "found  them  equally  effective  and  prefer  the 
variation",  although  their  comments  suggested  they  were  not  clear 
about  the  distinction  being  made.  Actually  there  were  very  few 
audio  exercises  among  the  questions  so  it  is  likely  that  they  may 
not  even  have  been  aware  of  them.  They  seemed  to  interpret  the 
question  as  referring  to  the  text  with  audio  feature  of  the  IVD 
materials . 

Almost  no  one  indicated  they  either  often  or  regularly 
intentionally  selected  Incorrect  answers,  or  chose  alternate 
paths  through  the  system  In  order  to  learn  from  the  special  helps 
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provided.  On  the  other  hand,  students  Indicated  they  were  often 
able  to  grasp  the  general  meaning  of  the  video  presentations,  and 
found  the  presentations  to  be  interesting.  The  latter  finding 
was  especially  positive,  since  the  middle  response  of  "some 
interesting,  some  uninteresting"  might  have  been  a  seductively 
reasonable  response,  but  most  students  (28  of  35  respondents) 
found  the  materials  to  be  either  "interesting"  (N*23)  or  "very 
interesting"  (N=5). 

Factors  affecting  IVD  use.  Table  18  shows  the  importance  of 
various  factors  related  to  the  amount  of  IVD  usage.  The  first 
set  of  items  relate  to  why  they  used  the  system  as  much  as  they 
did,  and  the  second  part  refers  to  why  they  didn't  use  the  IVD 
more.  As  previously  shown,  most  of  the  general  characteristics 
of  the  IVD  system  are  positively  responded  to  by  the  students. 

The  realism  of  the  material  and  a  break  from  other  study  have 
especially  strong  appeal,  while  not  having  time  enough  for 
extensive  use  is  the  greatest  detractor.  Liking  or  disliking 
computers  is  not  an  important  factor  in  general,  but  can  be 
decisive  in  individual  cases. 


Table  18 

User  Assessed  Importance  of  Factors  in  Amount  of  IVD  Use. 


Unimportant 

1 

2 

3 

4 

5 

Important 

Why  Use  IVD 


Learn  with  visuals 

1 

1 

7 

19 

7 

(M-3.86) 

Quality  of  Materials 

1 

2 

5 

20 

7 

(M-3.86) 

Realistic  Context 

1 

0 

2 

15 

17 

(M-4.34) 

Like  Computers 

7 

6 

14 

6 

2 

(M-2.71) 

Quality  of  Equipment 

1 

4 

7 

16 

5 

(M-3.60) 

Break  Monotony 

1 

1 

1 

5 

12 

(M-4.30) 

Why  Not  Use  IVD 

Learn  with  other  Material 

6 

4 

11 

8 

5 

(M-3.09) 

Not  useful  to  pass  tests 

9 

6 

9 

3 

7 

(M-2.79) 

Distracting  procedures 

11 

12 

8 

1 

2 

(M-2.14) 

Not  enough  time 

5 

4 

3 

6 

16 

(M-3.70) 

Dislike  computers 

21 

5 

7 

1 

6 

(M-1.68) 
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IVD  user  assessment.  Turning  to  the  two  items  reflecting 
overall  assessment  of  the  IVD,  these  trial  users  very  definitely 
supported  continued  use  of  the  IVD  as  either  an  "excellent 
innovation  with  great  potential"  (N*15)  or  as  a  "useful  aid  which 
significantly  contributes  to  learning"  (N=l4).  Based  on  either 
the  written  or  voiced  comments,  this  recommendation  is  related  to 
the  kind  and  range  of  benefits  indicated  by  the  following  general 
areas  illustrated  by  sample  quotes:  (in  order  of  frequency  of 
comment  shown  in  parenthesis) 

1.  A  break:  "a  change",  "a  break",  "get  burned  out  on 
one  mode",  "entertaining",  "reward",  "recess",  "interesting"  (13) 

2.  Realism:  "a  good  cultural  acclimatizer  to  people, 
culture,  places,  and  speed  [of  speech]",  "realistic",  "hear  it  as 
they  speak  it",  "met  objectives  if  these  are:  exposure  to  the  way 
its  going  to  be  and  getting  the  gist"  (8) 

3.  Reinforcement:  "a  reinforcement  tool — not 
essential,  but  an  aid",  "a  check  on  myself"  (6) 

4.  Speaker  variation:  "variety  of  speakers",  "broader 
exposure"  (5) 


5.  Special  type  of  learning:  "a  different  way  of 
learning",  "an  additional  dimension",  "get  certain  things  only 
from  the  video"(5) 


6.  Listening  comprehension:  "sharpens  listening 
skills",  "helps  my  ear" (4) 

7.  Idioms:  "helps  with  idioms"  (3) 

8.  Linguistic  integrator:  "brings  things  together" 

(1) 


-passive 


9.  Flexibility:  "I  like  option  to  use  as  I  feel  like- 
and  superficial  or  active  and  in  depth."  (1) 


10.  Provides  context:  "context  provided  for  text"(l) 


On  the  negative  side,  major  additional  points  made  by  the 
supporters  of  the  system  as  well  as  from  the  detractors  include: 


1.  Program  integration:  "Doesn't  match  up  with 
requirements",  "no  structure  on  which  material  to  use  when", 
"should  modify  both  the  video  and  the  course  to  fit  together", 
"feels  like  I'm  taking  two  courses",  "I'm  not  sure  of  the  target 
of  the  videos"  (9) 


2.  Time:  "no  time  for  it",  "not  a  priority  of  the 
program",  "would  like  to  use  it  more  but  there  was  no  time"  (8) 

3.  IVD  slowness:  "good  system  but  too  slow",  (5) 
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4.  A  break  but:  "...gets  boring  quickly", 
"...probably  not  worth  the  money",  "...doesn't  do  anything  for 
me"  (3) 


5.  Headstart:  "its  a  mistake  to  substitute  it  for 
headstart"  (3) 

6.  Not  related  to  learning:  "reinforcement  but  no 
good  for  learning",  "wouldn't  use  it  if  not  pushed  to,  doesn't 
make  much  contribution"  (2) 

7.  Not  appropriate  for  this  course:  "use  it  for 
exported  training  at  intermediate  level",  "send  it  to  learning 
centers"(2) 

8.  Better  alternatives:  "cultural  information  good 
but  could  be  presented  deeper  by  video  tape"  (1) 

9.  Pad:  "Army  is  being  captivated  by  technology. 

It's  money  spent  in  the  wrong  way"  (1) 

Finally,  two  of  intriguing  suggestions  for  additional  video 
topics  were  made:  a  disc  on  cultural  "faux  pas"  and  preparation 
for  the  driving  test. 

Summary 

Comparison  of  language  proficiency  between  the  Baseline  and 
IVD  groups  showed  the  IVD  group  to  have  significantly  higher 
scores.  However,  because  of  a  change  in  the  final  exam  at  the 
time  of  the  field  test  and  the  presumed  impact  that  a  new  final 
exam  has  on  both  tachlng  and  learning,  it  is  not  possible  to 
separate  the  effects  of  the  changes  in  the  final  from  the  effects 
of  the  IVD.  The  most  that  can  be  said  is  that  the  new  final  and 
IVD  in  combination  increased  the  measured  proficiency  of  the 
German  Gateway  students.  The  higher  scores  of  the  IVD  group  are 
mainly  the  result  of  answering  more  of  the  questions.  Whether 
the  Field  Test  students  were  able  to  answer  more  questions 
because  they  were  better  able  to  understand  the  questions,  (for 
example  had  better  listening  comprehension),  were  more  willing  to 
try  to  respond,  or  simply  had  better  language  skills  cannot  be 
determined. 

Based  upon  the  number  of  course  modules  completed,  there  is 
no  evidence  for  this  research  to  suggest  that  the  IVD  contributes 
to  an  increase  in  the  amount  of  material  which  students  cover. 

On  the  other  hand,  to  the  extent  that  students  report  spending 
significantly  more  time  studying,  and  some  of  that  time  is 
involved  covering  the  IVD  material  which  is  new  material,  they 
may  in  fact  be  covering  more  material. 
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The  IVD  does  not  detract  either  from  the  time  the  students 
spend  with  the  instructors  or  from  the  student  assessed 
importance  of  that  time.  Instructors  remain  the  vital  part  of 
the  German  Gateway  course. 

The  IVD  material  does  not  appreciably  increase  or  decrease 
student  evaluation  of  the  German  Gateway  program,  which  remains 
high.  Rather  the  IVD  takes  its  place  along  with  other  course 
components  as  a  positively  valued  part  of  the  course.  The  only 
area  where  the  program  was  rated  significantly  different  was  the 
higher  Field  Test  rating  of  satisfaction  with  the  level  of 
listening  comprehension  achieved.  This  finding  combined  with  the 
fact  that  a  higher  percentage  of  Field  Test  students  rated 
listening  comprehension  as  one  of  the  easiest  skills  to  learn, 
while  more  Baseline  students  rated  listening  comprehension  as  one 
of  the  most  difficult  skills,  suggests  that  this  is  a  major  area 
of  IVD  impact.  Further  research  on  effects  of  the  IVD,  utilizing 
specific  measures  of  listening  comprehension,  would  be  important 
in  verifying  the  relationship,  as  well  as  clarifying  the  role 
listening  comprehension  has  in  overall  improved  proficiency. 

Generally  the  students  were  positive  in  their  overall 
reaction  to  the  IVD,  recommending  its  continued  use  in  the 
program.  They  found  the  materials  to  be  interesting  and  to 
offer  a  constructive  break  in  their  regular  study.  They 
especially  liked  the  exposure  to  a  variety  of  German  speakers, 
the  realistic  context  provided,  the  quality  of  the  materials,  and 
the  Inclusion  of  a  visual  learning  mode  in  the  program.  After 
the  first  few  lessons,  students  were  generally  able  to  get  the 
gist  of  the  presentations.  They  were  adequately  oriented  to  the 
system,  found  the  equipment  easy  to  use,  and  the  procedures  did 
not  detract  from  the  learning.  Generally,  attitudes  about 
computers  did  not  seem  to  be  an  important  factor  in  assessments 
of  the  system. 

On  the  negative  side,  students  and  staff  alike  thought  that 
Headstart  was  still  needed  in  the  program,  while  at  the  same  time 
feeling  they  also  need  more  time  to  spend  on  the  IVD.  Many 
Identified  a  lack  of  clarity  with  how  the  materials  fit  into  the 
program  indicating  a  need  for  a  clearer  definition  of  the  role  of 
the  IVD  in  the  program. 

The  text  and  audio  presentation  of  the  visual  material  and 
the  vocabulary  helps  were  the  most  popular  of  the  special 
features  of  the  program,  followed  by  the  grammar/cultural  notes, 
questions  and  sub-segment  selection  option.  The  ability  to 
replay  the  video,  have  keywords  or  phrases  highlighted,  or  access 
helps  while  working  with  the  questions  were  not  regarded  as 
especially  important  features.  Further,  students  did  not  try  to 
U3e  the  help  that  came  from  wrong  answers.  The  strongest 
criticisms  were  that  there  were  no  English  glossary  or 
translations  provided,  and  that  the  system  response  was  too  slow, 
taking  too  much  of  the  very  limited  time  they  had  for  IVD  use. 
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RESULTS:  OBSERVATIONS 


Observation  process 

A  single  observer  (AE)  conducted  on-slte  observations  for  the 
major  part  (March  through  May  1986)  of  the  field  test  on  a  half¬ 
time  basis  by  alternating  morning  and  afternoons  on  successive 
days  of  the  week.  The  observation  process  consisted  of  making 
periodic  notations  of  the  number  of  systems  in  use,  patterns  of 
seme  users  as  they  became  established,  and  other  characteristics 
related  to  IVD  use.  Comments  by  both  faculty  and  students 
directed  to  the  observer,  as  well  as  overheard  comments  were 
recorded.  With  student  permission,  a  small  sample  of  students 
was  systematically  observed  throughout  an  IVD  study  session. 

These  observations  consisted  of  noting  how  the  students  proceeded 
through  the  lesson,  and  other  behaviors  such  as  taking  notes  or 
making  flash-cards.  Similarly,  three  instructors  were  observed 
and  questioned  as  they  reviewed  some  of  the  lessons,  to  discover 
not  only  how  they  chose  to  proceed  through  the  lessons,  but  also 
what  they  looked  for,  any  criticisms,  ideas,  and  specifically 
noled  materials.  During  the  last  six  weeks  (June  through  mid- 
July),  the  time  spent  in  observations  was  greatly  reduced  both 
because  there  were  few  persons  In  the  program  and  because  it 
became  more  difficult  to  perform  the  observations  due  to  nature 
of  the  changed  facility,  which  will  be  discussed. 

What  follows  is  based  on  information  gathered  from 
observations.  The  observation  process  itself  determined  the 
content  of  the  information  gathered  to  a  great  extent.  That  is, 
the  process  resulted  in  the  recognition  of  issues,  patterns,  and 
questions  which  then  became  the  basis  for  subsequent  observations 
and  interview  topics.  In  this  way,  the  context  within  which  the 
field  test  took  place,  and  the  many  factors  that  had  an  impact  on 
the  field  test  situation,  became  apparent. 

Description  of  the  German  Gateway  field  test 

The  field  test  of  the  IVD  materials  in  the  German  Gateway 
program  divides  into  two  distinct  phases  and  will  be  presented 
accordingly.  The  phases  are  primarily  distinguished  by  the 
location  of  the  program,  but  as  will  be  evident,  location 
incorporated  many  other  relevant  features.  Typically,  each  year 
during  the  peak  student  input  periods,  the  Gateway  program  must 
expand  its  operations  beyond  the  normal  Gateway  site.  This  year, 
the  program  moved  part  of  its  operations  to  a  former  elementary 
school  now  used  by  DLI .  The  Lighthouse  School  building  is 
located  on  a  residential  street  in  Pacific  Grove,  California, 
about  two  miles  away  from  the  main  DLI  campus.  The  Gateway 
building,  on  the  other  hand,  is  a  small  barracks-like  building 
located  on  the  Presidio  of  Monterey,  used  exclusively  by  the 
German  Gateway  program.  Students  who  started  at  the  Lighthouse 
building  completed  the  entire  program  there.  The  transition  back 
to  the  Gateway  facility  took  place  by  having  new  students  start 
the  course  there  rather  than  by  moving  anyone. 
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The  Headstart  portion  of  the  program  was  dropped  in  order  to 
have  more  time  for  the  IVD  materials.  This  meant  that  the 
instructors  had  to  devise  a  means  of  incorporating  the  material 
that  would  normally  have  been  covered  by  Headstart  into  the 
Gateway  course. 

Phase  I:  Shakedown 


The  first  phase,  consisting  of  two  student  inputs  for  a  total 
of  25  subjects,  took  place  under  unique  conditions  at  the 
Lighthouse  School  facility.  For  about  the  first  week,  it  was  as 
much  a  "shakedown"  of  the  German  Gateway  program  operating  in  a 
different  environment  and  under  different  conditions  as  it  was  of 
the  IVD.  After  some  initial  chaos  created  by  the  move,  crowding, 
lack  of  heat  and  other  minor  amenities,  the  program  seemed  to 
settle  into  a  smooth  and  comfortable  operation.  At  this  facility 
there  were  four  large  student  study  rooms  with  4  to  7  students 
per  room  with  two  video  stations  in  the  center  of  each  room. 
Noticeable  differences  in  both  study  and  IVD  use  behavior  were 
observed  among  the  rooms,  suggesting  the  operation  of  emergent 
norms  relative  not  only  to  the  IVD  system,  but  also  to  the  German 
Gateway  program  in  general. 

Students  were  told  they  were  participating  in  a  field  test, 
that  they  were  expected  to  use  the  IVD  materials  and  were  given  a 
one-on-one  orientation  to  the  IVD  system.  In  the  orientation  the 
students  were  encouraged  to  explore  the  system,  develop  their  own 
approaches,  and  to  use  the  system  in  whatever  way  worked  best  for 
them.  The  German  Subject  Matter  Expert  who  had  helped  develop 
the  German  Gateway  IVD  materials  now  shifted  roles  and  was  on 
hand  as  the  IVD  system  expert  to  orient,  trouble-shoot,  answer 
questions,  and  generally  support  and  encourage  the  use  and 
integration  of  the  IVD  in  the  program.  For  example,  as  the  time 
for  taking  the  midterm  examination  (an  informal,  ungraded  test  to 
give  students  feedback  on  how  they  are  progressing)  approached 
for  some  of  the  students,  he  posted  a  notice  Identifying  the  IVD 
lessons  related  to  the  topics  covered  by  both  the  midterm  and 
final . 

So  far  as  could  be  determined  from  the  general  observations, 
all  students  began  with  the  system  defined  approach.  That  is, 
they  3imply  proceeded  through  the  system  as  it  was  set  up: 
viewing  a  segment,  using  the  various  system  festures  pertaining 
to  the  lesson,  then  proceeding  on  to  the  next  segment  of  a 
program.  Later,  variations  began  to  emerge  from  Individual 
experimentation.  These  methods  seemed  to  develop  as  students 
tried  to  get  the  most  from  the  limited  time  they  allotted 
themselves  to  IVD  use. 

During  the  first  two  weeks  instructors  were,  at  times, 
observed  sitting  with  a  student  at  an  IVD  station,  often  with 
several  other  students  looking  on,  listening,  and  asking 
questions.  As  student  demands  of  Instructor  time  increased,  this 
phenomenon  ceased  and  students  worked  alone  at  the  IVD  stations. 
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IVD  use.  The  way  In  which  students  used  the  IVD  materials 
during  this  initial  phase  can  serve  as  a  basis  for  the 
development  of  categories  to  characterize  student  use  patterns. 
Five  discernable  patterns  were  Identifiable:  Non-users,  High 
Users,  Regular  Users,  Dependents  and  Others. 

Six  persons  were  essentially  non-users  of  the  IVD  system. 

One  was  simply  against  technology  and  was  strongly  opposed  to  the 
system  from  the  start.  Of  the  other  five,  three  did  not  use  the 
IVD  materials  at  all  after  the  first  couple  of  weeks,  with  the 
other  two  using  the  materials  only  occasionally.  All  five  shared 
a  similar  attitude,  regarding  the  IVD  material  as  possibly  useful 
if  they  had  the  time  for  it,  but  the  rest  of  the  course  was 
assumed  to  contain  the  "meat"  that  had  to  be  covered.  None  had 
prior  German  training  and  all  were  very  concerned  about  just 
getting  through  the  course.  Because  of  the  time  Involved  in 
using  the  IVD  materials,  they  considered  the  IVD  an  impediment 
rather  than  an  assist  in  reaching  their  goal. 

Four  of  these  persons  shared  a  room  at  the  end  of  the 
corridor  separate  from  the  central  flow  of  traffic.  They  evolved 
a  unique  arrangement  of  working  in  pairs  even  when  meeting  with 
Instructors.  In  essence,  they  developed  a  social  bond  which 
supported  their  learning  effort  and  included  normative 
definitions  of  both  the  course  and  the  IVD  materials. 

Two  persons  could  be  considered  high  users  of  the  IVD  by 
virtue  of  regular  and  extensive  use  of  the  system.  Both  began 
with  considerable  background  in  German  and  were  highly  motivated 
to  attain  as  much  functional  language  ability  as  possible.  One 
person  who  had  had  three  years  of  German  in  high  school  used  the 
books  and  audio  tapes  at  night  as  refresher  materials.  He 
concentrated  most  of  his  days  in  the  program  on  getting  as  much 
conversational  practice  as  possible  with  instructors  and  using 
the  IVD  materials.  The  use  pattern  of  the  second  person  was 
similar,  except  with  less  German  language  background,  he  spent 
somewhat  more  time  with  other  course  material. 

Regular  users  constituted  the  largest  user  category  in  the 
initial  phase.  These  persons  were  somewhat  intimidated  by  the 
first  IVD  lessons,  so  were  slow  to  begin  use  of  the  IVD  and  did 
so  only  with  Instructor  urging.  They  generally  used  the  videos 
at  the  completion  of  the  rest  of  their  work  on  each  module  as 
instructors  suggested.  The  instructors  recommended  viewing  the 
videos  as  review,  summary,  and  reinforcement.  IVD  use  picked  up 
as  these  students  progressed  through  the  course,  and  acquired 
more  of  a  language  base.  Passing  the  midterm,  as  well  as  their 
own  sense  of  a  growing  ability  to  use  German,  resulted  in  a  more 
relaxed  approach  and  more  use  of  the  IVD  which  was  referred  to  as 
"recess",  "a  reward",  "the  fun  stuff." 

The  regular  users  exhibited  characteristics  of  both  non¬ 
users  and  high  users.  At  the  beginning  when  they  were  concerned 
about  their  ability  to  get  through  the  course  and  intimidated  by 
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the  task  of  learning  the  language,  they  concentrated  on  the 
standard  program  which  was  also  a  more  familiar  learning  mode. 

As  they  developed  some  facility  and  security  in  the  language, 
they  were  able  to  focus  on  using  the  language  in  real  contexts, 
turning  more  to  the  IVD  materials  in  the  process. 

Two  of  the  regular  use  students  were  of  particular  interest 
as  the  only  ones  who  managed  to  do  what  many,  including  non¬ 
users,  said  they  intended  to  do.  They  finished  the  regular 
program  and  final  exam  several  days  be. 'ore  they  were  due  to 
depart,  spending  the  final  days  on  the  IVD  materials.  In  fact 
they  succeeded  in  keeping  "their"  system  when  movers  came  to  move 
it  to  the  Gateway  building.  They  were  the  only  students  who  not 
only  had  a  chance  to  complete  all  video  materials  but  also  went 

back  and  reviewed  earlier  lessons  as  a  way  of  unifying  the  course 

material  with  situations  they  would  soon  be  confronting. 

One  regular  user  could  almost  be  considered  to  represent  the 
ideal  in  approach  to  the  course  in  general  and  the  IVD  in 
particular.  His  background  in  Educational  Technology  gave  him  a 
clear  understanding  of  the  role  of  such  materials.  The  various 
components  of  the  course  seemed  to  be  accepted  on  a  roughly  equal 
basis,  all  relevant,  useful  and  used.  He  was  one  of  the  few  who 
did  not  voice  any  problem  with  lack  of  integration  of  the  IVD 

material  into  the  rest  of  the  program,  nor  did  he  get  caught  up 

in  the  question  of  whether  these  materials  were  a  good  idea  or 
not . 


Dependents  in  the  German  Gateway  program  have  a  somewhat 
unique  situation.  They  do  not  have  the  pressure  that  their 
sponsors  do  to  complete  the  course  with  an  acceptable  grade  to  go 
on  their  records.  On  the  other  hand,  they  do  have  incentive  to 
achieve  real  functional  ability.  As  one  dependent  put  it: 

"We're  the  ones  that  will  actually  be  doing  most  of  this  kind  of 
stuff"  (handling  the  kinds  or  situations  covered  in  the  course). 

The  dependents  were  also  handled  differently  in  the  program 
during  the  field  test.  They  went  through  the  headstart  materials 
first,  then  moved  through  as  many  Gateway  modules  as  they 
could.  Several  of  the  dependents  spent  much  of  their  last  week 
or  two  working  with  the  IVD  materials,  finding  them  to  be  the 
most  useful  to  their  perceived  needs. 

Two  other  persons  who  do  not  fit  into  any  of  the  above 
categories  are  worthy  of  note  because  their  circumstances  suggest 
other  important  issues.  The  first  is  a  person  who  had  to 
complete  the  program  in  four,  rather  than  seven  weeks.  As 
someone  with  extensive  computer  background  he  began  using  the  IVD 
materials  immediately,  intrigued  as  much  by  the  idea  as  by  the 
content.  However,  after  his  initial  effort  he  concentrated  on 
the  other  materials  using  the  IVD  system  very  little  until  he 
completed  his  final.  He  then  spent  about  two  days  going  through 
as  many  of  the  IVD  lessons  as  he  could.  This  person  served  to 
draw  attention  to  the  potential  relevance  of  comfort  or 


discomfort  with  computers  as  an  intervening  factor  in  the  use  of 
the  IVD.  However,  subsequent  observation  and  interview 
information  suggests  that  neither  strong  computer  background  nor 
initially  voiced  concerns  about  inexperience  with  computers  had 
significant  impact  on  use  of  or  attitude  about  the  IVD  for  the 
rest  of  the  students.  This  coincides  with  the  finding  from  the 
questionnaire  data  that  like  or  dislike  of  computers  was  not  an 
important  factor  in  student  assessments  of  the  IVD.  Computer 
background  is  not  a  variable  on  which  systematic  data  was 
collected  in  this  study,  but  might  be  a  variable  worth  examining 
in  future  research. 

The  second  person  falling  into  the  other  category  is  relevant 
in  demonstrating  a  potential  misuse  of  the  IVD.  This  person 
spent  the  first  weeks  in  the  course  working  on  the  IVD  to  the 
exclusion  of  other  course  material.  Until  his  instructor 
intervened,  he  was  going  through  the  modules  quite  superficially, 
covering  several  lessons  a  day.  Without  a  background  in  German, 
he  was  probably  gaining  very  little  functional  ability.  Put  on 
notice  that  he  was  Jeopardizing  his  chances  of  completing  the 
program,  he  began  to  use  other  course  materials  to  the  exclusion 
of  the  IVD,  using  it  very  little  after  the  high  use  start. 

Initial  phase  summary.  Non-users  were  those  who  tried  the 
IVD  system  a  few  times  early  in  the  program,  then  used  it  very 
rarely  or  not  at  all  for  the  duration.  In  this  first  phase 
group,  all  the  non-users  were  zero  level  beginners.  High  users 
typically  averaged  2  to  4  hours  per  day  working  with  the  IVD 
materials.  They  had  fairly  strong  backgrounds  in  German  and  used 
the  IVD  materials  to  refine  and  extend  solid  linguistic 
ability.  Regulars  were  the  students  who  generally  worked  with 
the  IVD  lessons  for  each  module  as  they  went  along.  Some 
completed  the  basic  module  materials  then  viewed  the  videos, 
others  turned  to  the  videos  as  a  break  in  the  process  of  covering 
a  module.  In  the  beginning,  they  were  somewhat  intimidated  by 
the  IVD  materials  and  initial  use  was  low.  As  the  course 
progressed  their  use  of  the  IVD  picked  up  and  became  a  regular 
part  of  their  routine.  The  position  of  the  dependents  in  the 
program  was  initially  confused  and  they  were  uncertain  about 
using  the  IVD  system.  They  began  as  non-users  while  covering 
Headstart,  gradually  tested  the  system  as  they  progessed  through 
German  Gateway  Course,  and  ended  as  high  users.  Others  were 
those  who  exhibited  unique  patterns  for  individual  reasons  and 
therefore  do  not  readily  fit  Into  the  above  categories. 

For  the  most  part  the  general  attitude  of  persons  in  this 
first  group  towards  the  IVD  materials  was  quite  positive,  even 
among  the  non-users.  They  liked  having  the  visual  material  as  a 
break  from  text  and  audio  tapes,  and  they  appreciated  the 
realistic  perspective  provided  on  the  situations  they  would  be 
encountering  in  Germany. 
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The  most  prevalent  complaint  was  that  the  system  help 
features  were.  In  fact,  of  little  help.  When  definitions  and 
explanations  were  provided  in  German,  the  helps  were  often 
understood  little  better  than  what  was  being  defined  or 
explained.  The  students  were  then  forced  to  turn  to 
dictionaries,  instructors,  or  "Just  forget  it."  They  found  this 
to  be  too  time-consuming  and  frustrating.  Eventually  many  ceased 
to  use  these  helps.  Some  of  the  non-users  claimed  this  was  their 
major  objection  to  using  the  system. 

The  second  common  complaint  was  that  the  IVD  materials  were 
not  integrated  into  the  rest  of  the  course.  Students  often  did 
not  find  a  fit,  by  topic,  vocabulary,  or  structure  between  the 
course  modules  and  the  IVD  materials  designed  to  accompany 
them.  One  or  two  students  were  able  to  discern  the  difference  in 
underlying  pedagogies  and  felt  caught  in  a  conflict  between 
them.  There  was  no  consensus  on  how  the  IVD  could  or  should  be 
integrated  into  the  program.  There  was  simply  a  pervasive 
recognition  that  there  was  no  integration. 

Both  students  and  staff  expressed  a  need  for  a  better 
matching  between  the  ability  levels  of  students  and  that  required 
by  the  IVD  materials.  The  IVD  materials  maintained  a  constant 
language  level  while  student  abilities  changed.  The  result  was 
that  the  first  lessons  were  perceived  as  too  difficult  and  the 
later  lessons,  especially  the  questions,  as  "insultingly"  easy. 
The  difficulty  of  the  early  lessons  heightened  the  anxiety  or 
some  students,  and  although  no  long-term  problems  resulted,  one 
instructor  reported  having  to  expend  considerable  effort  to 
offset  this  effect. 

The  unique  situation  at  the  Lighthouse  facility  gave  rise  to 
more  of  a  social  context  than  typically  occurs  in  Gateway  with 
the  result  that  there  developed  normative  definitions  related  to 
both  the  program  and  the  IVD  system.  The  arrangement  there 
facilitated  observation  and  easy  non-interference  observer  access 
to  students  and  staff  to  pick  up  comments,  ask  questions,  and 
conduct  on-the-spot  interviews  on  an  on-going  basis. 

There  was  no  detectable  general  attitude  toward  the  IVD 
materials  on  the  part  of  any  instructor.  In  line  with  the 
official  field  test  policy  they  regularly  suggested  that  students 
view  the  videos  upon  completion  of  a  module  as  reinforcement  or 
as  a  break  before  going  on  to  the  next  course  module.  The  non¬ 
users  were  encouraged  but  not  pushed  to  use  the  IVD.  In  this 
context  the  individual  use  patterns  described  above  could  emerge. 

Interviews  were  conducted  with  instructors  as  this  phase 
ended.  They  expressed  generally  positive  reactions,  though  were 
neither  strongly  enthusiastic  nor  especially  critical.  In 
contrast,  the  question  of  whether  they  thought  the  materials 
would  be  useful  in  the  DLI  German  Basic  Course  elicited  very 
definite  enthusiasm.  Every  instructor  described  in  some  detail 
how  he  or  she  would  like  to  use  the  videos  in  the  Basic  Course. 
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A  problem  was  Identified  In  response  to  a  direct  interview 
question  about  the  Impact  of  dropping  Headstart.  The  instructors 
seemed  to  accept  that  they  had  to  make  up  for  some  of  the  missing 
background,  but  did  not  communicate  this  as  a  particular 
problem.  However,  one  instructor,  involved  in  administering  the 
Oral  Proficiency  Examination,  did  note  that  there  were  gaps  in 
the  students'  backgrounds  in  some  basic  areas  that  showed  up  even 
at  the  end  of  the  course — a  fact  that  suggests  it  might  be 
difficult  for  each  instructor  to  insure  that  all  Headstart 
material  is  somehow  incorporated. 

Phase  II:  Settling  in 

The  transfer  of  the  program  back  to  the  German  Gateway 
building  offered  a  chance  to  see  what  differences,  if  any, 
occurred  as  the  IVD  was  settled  into  the  program's  home.  While 
the  initial  phase  group  finished  their  course  at  Lighthouse,  new 
students  were  entering  the  program  at  the  German  Gateway  building 
and  several  IVD  systems  were  moved  to  that  facility.  At  this 
point,  conditions  in  the  field  test  changed.  Students  continued 
to  be  informed  of  the  field  test  and  provided  one-on-one 
orientations,  but  the  orientation  Itself  changed  somewhat, 
benefiting  from  the  earlier  experiences .  The  rationale  for  the 
absence  of  English  was  more  fully  explained;  comments  from 
previous  users  could  now  be  quoted,  for  example,  "This  seems  to 
bring  it  all  together."  Examples  of  how  students  might  choose  to 
use  the  system  were  described  based  on  what  previous  users  had 
actually  done. 

The  normal  course  of  Gateway  instructor  rotations  and 
temporary  assignments  resulted  in  more  variation  in  instructional 
staff  in  this  phase.  Three  new  instructors  were  brought  in  for 
the  first  weeks  at  Gateway  until  the  last  Lighthouse  students 
finished  the  program  and  the  instructors  there  were  free  to  make 
the  transfer.  Two  of  the  original  four  field  test  instructors 
left  the  program  while  a  third  rotated  out  four  weeks  later, 
leaving  only  one  of  the  initial  phase  instructors  in  the  program 
for  the  duration  of  the  field  test.  One  of  the  new  Instructors 
continued  in  the  program  and  one  of  the  initial  phase  instructors 
rotated  out  after  four  weeks.  Table  19  depicts  these  staffing 
changes . 
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Table  19 

Field  Test  Instructor  (A-G)  Location  (L*Lighthouse,  G=Gateway) 
by  Month  (March-July  1986) 


M-/ -/-/-/ A-/-/-/-/M-/-/-/-/-/J-/-/-/-/J-/-/-/ 
A—  LLLLLLLLLLLLLLLGGGGGGGGGGGGGGGGGGGGGGGGGGGGG 
B—  LLLLLLLLLLLLLLLGGGGGGGG 
C—  LLLLLLLLLLLLLLL 
D—  LLLLLLLLLLLLLLLLLLL 

E—  GGGGGGGGGGGGOGOGGGGGGGGOGGGGGGGGGGG 

F—  GGGGGG 

G—  GGGG 


Upon  review  of  some  of  the  IVD  materials,  the  one  new 
Instructor  who  continued  In  the  program  expressed  the  opinion 
that  they  were  good  videos  but  too  advanced  for  beginning 
students,  that  the  system  was  too  slow,  and  that  the  lessons 
simply  took  too  much  time  in  such  a  short  intensive  course. 

A  major  change  in  the  program  occurred  around  use  of  the 
Headstart  materials  which  had  been  dropped  from  the  program  at 
the  beginning  of  the  field  test  to  allow  time  for  use  of  the  IVD 
materials.  All  three  new  instructors  voiced  reservations  about 
the  ability  of  students  to  handle  the  Gateway  course  without  the 
Headstart  background.  Students  were  given  the  Headstart 
materials  to  work  with  on  their  own  in  the  evenings.  Actually, 
however,  the  major  part  of  the  first  1  to  2  weeks  of  instruction 
dealt  with  the  Headstart  material. 

There  were  not  only  changes  in  the  nature  of  the  program  as 
It  returned  to  the  Gateway  facility  but  also  in  the  observation 
process.  Observations  were  more  difficult  as  students  had  their 
own  rooms,  worked  behind  closed  doors,  with  some  of  the  IVD 
stations  located  in  student  rooms.  Talking  with  students,  as 
well  as  instructors,  became  more  intrusive  and  there  was  less 
opportunity  to  hear  spontaneous  comments  and  interchanges  which 
could  reveal  the  tone  or  general  atmosphere.  Further,  it  was 
usually  not  possible  to  determine  the  total  number  of  stations  in 
use  at  any  one  time. 

The  same  conditions  that  curtailed  the  observation  process 
also  impacted  on  the  program  with  a  decrease  in  the  social 
context  of  learning  that  had  developed  at  Lighthouse  facility. 
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There  were  no  strongly  supported  group  attitudes  affecting  IVD 
use,  nor  normative  patterns  supporting  specific  study 
activities.  Nevertheless  a  certain  amount  of  observation  was 
possible,  permitting  the  tentative  conclusions  which  follow. 

IVD  use.  The  first  inputs  at  Gateway  seemed  to  follow  the 
pattern  of  regular  users  of  the  initial  group:  slow  start  on  the 
IVD  while  students  became  more  comfortable  with  the  course  and 
the  language,  then  their  use  of  the  IVD  materials  picked  up. 

They  seemed  to  accept  these  materials  raatter-of-factly,  as  Just  a 
part  of  the  course.  There  were  also  some  very  low  or  non-users, 
most  of  whom  intended  to  use  the  IVD  materials  at  the  end  of  the 
course  and  often  never  got  around  to  doing  so.  There  were  no 
apparent  high  users  in  this  group.  Two  dependents  continued  the 
pattern  of  regular  to  high  IVD  use. 

Later  student  inputs  in  this  second  phase  demonstrated  a 
marked  reduction  in  IVD  use.  Many  students  were  not  encouraged 
to  use  the  IVD  materials  until  about  halfway  into  the  program, 
then  often  simply  did  not  find  time  for  it.  Information  from  the 
baseline  student  interviews  suggests  that,  roughly  midway  into 
the  program,  students  realize  they  cannot  "do  it  all"  and  begin 
to  limit  their  expectations  of  what  they  will  be  able  to  cover. 
This  is  a  rather  unpropitious  time  to  introduce  a  new  element 
with  which  students  have  had  little  or  no  previous  experience. 

It  would  probably  be  better  to  follow  the  earlier  practice  of 
introducing  students  to  the  IVD  materials  and  encouraging  their 
use,  "as  a  break"  at  least,  from  the  beginning.  The  system  then 
would  be  familiar  when  later  it  became  a  highly  reccommended  part 
of  the  course. 

Second  phase  summary.  During  the  second  phase  of  the  field 
test  the  Headstart  issue  emerged  as  a  focal  one  with  most 
students  voicing  the  opinion  that  Headstart  was  an  absolute 
requisite  for  Gateway  for  anyone  with  no  background  in  German. 
Complaints  about  the  lack  of  English  in  the  IVD  materials 
persisted.  The  issue  of  integration,  while  still  sometimes 
recognized,  was  less  relevant  compared  to  the  Headstart  issue. 

An  additional  common  criticism  voiced  by  this  group  was  about  the 
slowness  of  the  system,  both  in  the  loading  process  and  in 
responsiveness.  There  was  also  general  agreement  that  the  IVD 
materials  were  too  advanced  to  be  of  any  use  until  somewhere 
between  Modules  4  and  6  of  the  German  Gateway  course. 

Instructors  were  more  variable  in  the  second  phase  both  in 
total  number  of  instructors  in  the  program  and  in  changes  in  the 
instructors  working  with  particular  students.  Reflecting  the 
rotations  of  instructors  into  the  program  and  the  changes  in  the 
student  population,  this  fact  is  a  normal  part  of  the 
program  that  happened  to  affect  the  second  phase  group 
differently  than 
the  first. 
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To  the  extent  that  the  first  subjects  during  this  phase 
followed  the  most  common  pattern,  that  of  regular  users  of  the 
initial  phase  group,  it  became  easy  to  assume  that  this  was  to 
become  the  more  or  less  established  pattern.  A  picture  began  to 
emerge  of  the  place  the  IVD  materials  would  probably  occupy  in 
the  German  Gateway  Program:  as  something  that  could  hone  and 
extend  the  abilities  of  the  more  advanced  student;  another 
learning  mode  and  a  "break”  for  the  average  student;  but  as  a 
dispensable  element  for  those  who  felt  overwhelmed  by  the  amount 
of  material  to  cover.  As  final  student  input  into  the  field  test 
and  subsequent  student  inputs  demonstrated  a  downward  trend  in 
the  use  of  the  IVD  materials,  that  picture  had  to  be  revised. 

By  the  end  of  the  field  test,  the  IVD  materials  were  being 
used  only  upon  completion  of  the  basic  11  modules  in  the 
course.  Comments  by  both  instructors  and  students  indicated 
there  was  little  program  encouragement  to  use  the  IVD 
materials.  As  one  student  put  it:  "It  doesn't  seem  structured 
to  use  them.  It's  structured  to  sit  here  and  work  on  the 
modules,  to  go  in  to  see  the  Instructor,  and  to  come  back  and 
work  on  the  modules  some  more.  There  isn't  time  for  anything 
else."  And  as  one  instructor  observed:  "You  better  do  something 
to  promote  your  system.  Hardly  anyone  is  using  it";  indicating  a 
lack  of  ownership  for  this  system.  Possibly  after  the  field 
test,  and  the  disappearance  of  the  "outsiders"  (observer  and  SME) 
this  attitude  may  change. 

Just  as  the  first  weeks  of  the  initial  phase  involved  a 
transition  into  the  field  test,  final  weeks  of  the  second  phase 
seem  to  involve  a  transition  out  of  the  field  test.  With  both 
the  system  expert  and  the  observer  spending  much  less  time  on¬ 
site,  the  visibility  of  a  field  test  taking  place  was 
diminished.  Along  with  the  end  of  the  field  test  was  an  apparent 
end  to  the  policies  instituted  for  the  field  test.  Students  in 
the  German  Gateway  program  but  not  in  the  field  test  neither  were 
being  systematically  oriented  to  the  IVD  system  nor  were  told  to 
use  the  IVD  materials  before  going  onto  the  next  modules.  The 
IVD  materials  were  simply  available  for  use. 

IVD  system  use.  The  most  IVD  stations  observed  in  use  at 
once  was  six  (of  nine  available)  for  25  students,  although  on  two 
occasions  the  observer,  arriving  in  the  afternoon,  was  told  by 
both  students  and  staff  that  all  systems  had  been  in  use  the 
previous  morning.  "You  should  have  been  here  this  morning,  I 
couldn't  find  a  video  open,"  a  student  reported.  At  the  other 

extreme,  there  were  numerous  occasions  when  there  were  no  IVD 

stations  observed  in  use  for  days  at  a  time.  The  average  number 

of  IVD  stations  in  use  per  observation  was  Just  under  three  for 
the  first  phase  and  slightly  over  one  for  the  second  phase.  As 
already  indicated,  the  location  of  the  units,  the  use  patterns  of 
the  particular  students  in  the  program  at  the  time,  and  the 
orientation  of  the  program  towards  using  the  IVD  materials  are 
the  main  determinants  of  the  number  of  units  required  by  the 
prcg  am.  If  the  pattern  of  the  regular  users  became  the  common 
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one,  about  one  IVD  station  per  3  to  5  students  is  probably 
adequate.  For  example,  the  two  stations  per  student  room  of  4  to 
7  people  at  the  Lighthouse  facility  seemed  to  work  out  well. 
Having  stations  in  individual  student  rooms  as  was  done  at  the 
Gateway  facility  discourages  use  by  any  student  other  than  the 
room  occupant,  and  would  almost  require  a  unit  per  student. 

Fitting  the  IVD  materials  into  the  German  Gateway  program 

The  explicit  definition  of  the  role  of  the  IVD  materials  in 
the  German  Gateway  program  is  as  a  "supplement"  to  the  existing 
program  with  the  purpose  of  "reinforcing"  the 
learning/instruction  of  the  rest  of  the  program.  The  fact  that 
both  terms  are  regularly  used  by  staff  and  students  alike  with 
reference  to  the  IVD  materials  would  seem  to  suggest  the 
existence  of  a  consensus  on  how  these  materials  fit  into  the 
program.  There  are,  however,  variations  in  the  meaning  of 
"supplement"  as  the  concept  is  translated  into  practice.  There 
is  also  a  lack  of  clarity  around  the  meaning  of  "reinforcement". 

Supplement,  as  a  concept  put  into  practice,  seemed  to  have 
any  or  all  of  the  following  possible  meanings  depending  on  the 
situation,  the  person,  or  the  perspective. 

1.  Similar  to  the  German  language 
magazines  and  newspapers  available 
in  the  lounge;  that  is,  something 
separate  from  and  external  to  the 
course,  which  stimulates  interest 
and  provides  a  challenge  to  learning 
but  is  not  part  of  the  program  of 
instruction.  It's  a  realia  for 
casual  use  as  time  and  Interest 
dictate.  As  the  field  test  ended, 
it  appeared  that  the  German  Gateway 
program  was  returning  to  basically 
the  pre-field  test  status  where  the 
course  is  conceived  as  the  set  of 
module  material  and  personal 
instruction,  except  that  the  IVD 
materials  are  available  for  summary 
or  overview  as  time  permits  .  Non¬ 
users  also  demonstrated  this 
conception  during  the  field  test. 

2.  As  something  appropriate  for  the 
student  with  some  background  in 
German,  to  stimulate  and  enhance 
their  learning  beyond  the  program's 
normal  offerings.  It  extends  the 
range  of  material  available.  The 
awareness  that  zero  level  beginners 
are  threatened  by  the  IVD  materials 
and  that  the  more  advanced  students 


41 


make  the  most  use  of  them,  results 
In  this  as  an  unintended  but 
emerging  conception.  The  practice 
of  encouraging  use  only  after 
students  have  completed 
approximately  half  the  course  also 
promotes  this  definition. 

3.  As  a  specialized  element 
contributing  uniquely  to  the 
teachlng/learnlng  process.  It  is  an 
essential  part  of  the  course  to  be 
used  regularly  along  with  other 
course  materials.  The  field  test 
policy  of  requiring  that  students 
view  the  videos  as  part  of  their 
coverage  of  a  module  supported  this 
definition  as  did  the  actual  use 
patterns  of  the  majority  of  field 
test  students.  This  conception  is 
not  sustained  without  its 
specification  as  policy  and  possibly 
other  support. 

4.  As  the  central  component  of  the 
course,  to  which  other  parts  of  the 
course  should  be  adapted.  It  is  the 
element  around  which  the  program 
should  be  organized.  Only  the 
suggestions  of  some  of  the  students, 
in  the  context  of  how  the  IVD  should 
be  integrated  into  the  program,  and 
the  use  patterns  of  the  highest 
users  supported  this  conception. 

Reinforcement  can  also  have  a  variety  of  meanings. 
Specification  of  the  "what"  and  "how"  of  reinforcement  is 
needed.  "What"  may  be  considered  the  overall  program,  specific 
components  (i.e.  instructors,  existing  text,  audio,  and  test 
materials),  or  specific  student  linguistic  skills.  Logically, 
then  the  place  of  the  IVD  materials  in  the  program  would  be 
dictated  by  defined  learning,  program  or  instructional 
objectives.  The  variation  in  the  operational  definitions  of 
"supplement"  indicates  that  there  is  also  no  operational 
consensus  on  objectives  for  the  IVD. 

DISCUSSION  AND  RECOMMENDATIONS 

The  German  Gateway  project  demonstrates  several  possibilities 
for  using  a  particular  set  of  IVD  materials  to  support  a  short 
term  intensive  foreign  language  instructional  program.  There  is 
evidence  of  a  range  of  conceptions  as  to  how  the  IVD  materials 
should  fit  into  the  program,  what  they  can  contribute  to  language 
learning  and  how  they  are  to  be  used  by  students  and  instructors 
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to  achieve  various  objectives.  This  situation  provides  useful 
information  for  purposes  of  research  and  evaluation  but  is 
confusing  to  students  and  staff  alike.  What  is  now  needed  is 
consensus  on  a  central  definition  of  how  to  make  the  best  use  of 
these  materials  in  support  of  program  objectives,  and  the 
formulation  of  program  policies  to  operationalize  that 
definition. 

The  German  Gateway  project  also  provides  a  basis  for 
identifying  some  of  the  difficulties  involved  in  the  adoption  of 
IVD  into  an  existing  program.  Central  problems  concern  questions 
of  equipment  security,  maintenance,  and  training;  and  integration 
of  the  new  materials  into  existing  curricula.  Failure  to  address 
these  questions  limits  students'  use  and  results  in  probable 
short-term  program  utility  of  the  IVD  materials. 

From  a  cost-effectiveness  perspective  it  makes  a  great  deal 
cf  difference  whether  IVD  materials  will  be  an  auxiliary  realia 
feature  in  a  program  used  by  a  few  students  once  in  a  while  or  an 
integral  part  of  a  program  used  by  most  students  on  a  regular 
basis.  Regardless  of  what  final  use  is  made  of  the  IVD  materials 
in  the  German  Gateway  program,  the  experience  to  date  can  be  used 
to  identify  some  general  recommendations  for  effectively 
incorporating  IVD  techology  into  an  instructional  program. 

The  following  recommendations  are  based  only  upon  the  German 
Gateway  program  but  provide  a  general  framework  for  identifying 
relevant  factors  for  consideration  in  planning  for  implementation 
and  maintenance  of  IVD  innovations  in  any  instructional  program. 

Supports 

Orientation  to  and  maintenance  of  the  IVD  system.  Some 
possibilities  are:  a  program  to  train  instructors  to  trouble¬ 
shoot  the  IVD  equipment  and  to  orient  students;  Identification  of 
someone  to  perform  these  functions  on  an  on-going  basis;  or 
possibly,  replacing  the  personal  orientation  with  packages 
similar  to  the  German  Gateway  modules  containing  the  diskettes, 
instructions,  learning  objectives,  and  ideas  for  alternative 
modes  of  using  the  system.  In  addition,  a  system  of  logistics 
for  replacing  diskettes  and  performing  any  other  routine 
maintenance  functions  needs  to  be  established. 

Instructor  orientation.  Provision  of  time  for  instructors  to 
become  familiar  with  the  IVD  materials  is  needed.  It  Is 
estimated  that  simply  viewing  all  of  the  videos  (without  using 
any  of  the  special  functions)  would  require  2  to  3  days.  Options 
might  include  allotting  time  to  new  Gateway  instructors  prior  to 
beginning  in  the  Gateway  program  or  reducing  student  load  to 
provide  time  on  a  daily  basis. 

Development  of  an  Instructor's  manual.  Such  a  manual  might 
include  "keywords"  in  the  system,  interesting  cultural  features, 
new  idioms  and  vocabulary,  and  Indications  of  where  IVD  language 


usage  differs  from  that  presented  In  the  regular  course.  These 
are  some  of  the  things  Instructors  looked  for  as  they  previewed 
the  IVD  materials.  Other  Inclusions  could  be  developed  according 
to  results  of  discussions  with  instructors. 

Staff  development .  Training  devoted  to  examining  and 
developing  ways  for  instructors  to  work  directly  with  IVD 
materials.  The  field  test  dealt  with  the  IVD  only  as  a  self- 
study  method.  Yet  the  possibility  of  profitable  instructor 
mediated  interaction  with  these  materials  with  one  or  more 
students  is  suggested  i*y  the  interest  and  experience  of  the  first 
days  of  the  field  test  and  the  request  by  students  for  more 
conversational  practice.  Using  the  IVD  system  as  a  base  for 
developing  practice  dialogues  and  discussion  could  expand  the 
skills  supported  by  the  system  to  include  speaking. 

Program/pollcy  changes 

Headstart .  As  already  described,  the  field  test  entailed  two 
different  policies  related  to  Headstart,  neither  of  which  seemed 
satisfactory.  Eliminating  Headstart  completely  left  gaps  in 
students'  preparation  for  Gateway.  Leaving  the  full  program  in 
the  course  Is  tantamount  to  insuring  students  will  not  have  time 
for  the  IVD  materials.  A  compromise  might  be  to  revise  the 
Headstart  portion  with  the  Intent  that  it  become  only  a  one  week 
program  for  zero  level  beginners. 

IVD  materials  and  German  Gateway  integration.  In  the  field 
test  the  responsibility  for  integrating  the  various  elements  of 
the  course  has  been  left  primarily  to  the  students.  The  student 
orientations  to  the  program  and  to  the  IVD  provide  them  with  some 
guidance  related  to  what  they  can  get  from  different  elements  of 
the  program.  However,  long  range  planning  could  appropriately 
include  revisions  of  the  German  Gateway  modules  and  instructor 
manuals  as  a  result  of  rethinking  the  overall  course  as  one  now 
including  IVD  materials  . 

IVD  requirement.  The  initial  phase  group  had  the  requirement 
to  view  assigned  IVD  materials  before  going  on  to  the  next 
module.  Pew  of  the  students  in  the  final  field  test  inputs  even 
knew  of  such  a  requirement.  There  is  no  question  that  sessions 
with  instructors  and  coverage  of  the  German  Gateway  modules  is 
required.  Yet  within  these  requirements  there  are  individual 
differences  in  where  students  concentrate  their  efforts  and  the 
time  students  actually  spend  with  instructors.  Similarly,  a 
policy  requiring  students  to  work  with  the  IVD  materials  would 
serve  to  integrate  the  materials  into  the  program,  while 
permitting  student  choice  and  variation. 

Physical  arrangements/security.  Issues  concerning  the 
physical  set-up  of  the  IVD  stations  were  identified  in  the  field 
test.  One  issue  relates  to  the  question  of  the  best  location  for 
optimum  student  use.  In  the  Lighthouse  facility  the  stations 
were  located  in  the  student  rooms  readily  available  and  visibly 


44 


used  (op  not  used)  by  others,  yet  were  away  from  the  distractions 
of  socializing  areas.  In  the  Gateway  facility  the  stations  were 
varyingly  located  in  the  lounge,  hallways,  or  individual  student 
rooms — that  is,  in  the  midst  of  movement  and  activity  or  readily 
usable  by  only  one  person.  Some  of  the  choices  of  locations  were 
dictated  by  security  measures  which  require  that  the  equipment 
not  be  easily  visible  from  the  outside.  In  general,  the  choice 
of  locations  for  the  IVD  stations  was  dictated  by  security, 
location  of  outlets,  and  what  rooms  were  available,  more  than  by 
any  question  of  how  to  facilitate  student  use. 

Modification  of  external  practices 


Headstart .  Pew  students  receive  Headstart  prior  to  arriving 
in  the  Gateway  program.  Pew  of  those  who  do  receive  it  actually 
study  it.  Efforts  to  change  both  the  distribution  system  and  the 
conception  that  Headstart  is  covered  in  the  program  could  help  at 
least  reduce  the  amount  of  time  needed  to  cover  the  material  in 
the  program. 

Longer  course.  Even  without  the  IVD  material  many  students 
have  said  the  course  should  be  longer.  Whether  there  is  any 
possibility  of  having  the  course  length  re-defined  by  lengthened 
assignment  could  be  explored. 
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Appendix  A 

GERMAN  CATEWAY  PRE-COURSE  QUESTIONNAIRE 
Purpose  of  Che  Questionnslre: 

The  following  questionnaire  Is  designed  to  gather  baseline  data  for  the 
Cerium  Cateway  progran.  In  order  to  tailor  our  course  to  best  fit  the  needs 
of  the  student,  we  arc  trying  to  get  a  reading  on  the  types  of  candidates  we 
get  In  our  prograa.  We  are  asking  you  to  help  In  this  effort  by  responding  to 
the  questions  which  follow.  YOU  NAME  IS  NOT  DESIRED  and  all  responses  will  be 
coablned  with  others  for  aggregate  statistical  analysis;  le.  THIS  IS  A  NON- 
ATTRIBUTABLE  QUESTIONNAIRE. 

Date  _  Research  Identifier 


(circle  the  appropriate  response) 

1.  Military  Status: 

a.  Military 

b.  Dependent 


2.  Age 

a.  21  -  30 

b.  31  -  40 

c.  41  -  50 

d.  51  -  60 

3.  Educational  Level: 

a.  High  school  or  less 

b.  Some  college  or  associate  degree 

c.  BA/BS  degree 

d.  MA/MS  or  higher  degree 

e .  Ph . D. 

• 

Based  on  experience  with  students  In  past  Gcraan  Gateway  courses  we  have 
tentatively  Identified  several  factors  related  to  course  success.  These 
Include:  exposure  to  Geraan  language  and  culture;  previous  training  In  German; 
previous  training  In  a  foreign  language.  The  following  questions  relate  to 
these  factors  as  they  apply  to  you.  The  Information  provided  will  aid  us  In 
verifying  the  relevance  of  these  factors. 

4.  Have  you  ever  been  to  Germany! 

a.  Yes 

b.  No 

If  yes,  for  hew  long? _  How  Recently? _ 

date 


5.  Have  you  ever  studied  German  In  any  formal  or  Informal  manner? 

a.  Yes 

b.  No 

(If  No  go  to  number  10) 
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6.  Was  your  training  In: 

a.  Home /Neighborhood 

b.  High  School 

c.  College 

d.  Graduate  School 

e.  Self-Study  Program 

f.  Other _ 

•peclfy 

7.  Was  this  training  In: 

a.  A  civilian  system 

b.  A  military  system 

c.  Both 

8.  What  was  the  approximate  number  of  semester  hours  or  high  school  years 

(Indicate  which)  of  the  above  mentioned  German  studies  that  you  participated 
In? _ 

9.  How  long  ago  did  you  finish  your  last  German  studies? 
a.  Within  the  last  year 

b*  Within  the  last  2  years 

c.  Within  the  last  5  years 

d.  More  than  5  years  ago 

10.  Have  you  ever  studied  another  foreign  language  In  any  formal  or  Informal 
manner? 

a.  Yes 

b.  No 

(If  No.  go  to  number  15) 

I 

11.  What  language(s)  have  you  studied?  (If  more  than  one.  lime  In  order  of 
most  to  least  fluent  language). 

1. _ 

2.  _ 

3.  _ 

(In  answering  questions  11-14,  write  the  name  of  the  language(s)  from  question 
10  above,  beside  the  appropriate  response) 

12.  Was  your  training  In: 

_ a.  Hons/Neighborhood 

_ b.  High  School 

_  c.  College 

_  d.  Graduate  School 

_  e.  Self-Study  Program 

_  f.  Other 
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13.  Was  your  training  In: 


_ a.  Civilian  systea 

_  b.  Military  ayataa 

_  c.  Both 

_  d.  Other 

14.  What  was  the  approximate  total  nuaber  of  semester  hours/high  school  years 
(indicate  which)  of  your  training  In  the  language(s)  studied? 


IS.  How  long  ago  did  you  finish  your  last  language  studies? 


a. 

Within 

the 

last 

year 

b. 

Within 

the 

last 

2 

years 

c. 

Within 

the 

last 

5 

years 

d. 

More  than 

3  years 

ago 

The  next  questions  are  designed  to  obtain  lnforaatlon  on  your  attitudes  and 
expectations  before  taking  thla  course.  For  each  item  below,  select  froa  the 
following  responses  the  one  which  best  expresses  your  reactions*  (Write  the 
appropriate  letter  In  the  blank  by  each  ltea) 

A.  Strongly  Agree 

B.  Agree 

C.  Disagree 

D.  Strongly  Disagree 

_  16.  Knowing  German  will  be  lnportant  In  performing  ay  official 

duties  In  Germany. 

_  17.  Taking  this  course  la  relevant  to  ay  Army  Career. 

_  18.  I  do  not  expect  to  have  any  trouble  learning  German* 

_  19.  Understanding  Geraan  culture  will  he  important  to  ae  while  In 

Germany. 

_  20.  I  need  to  learn  Geraan  In  order  to  function  In  German  society* 

(e.g.,  restaurants,  stores,  travel,  etc*) 
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After  completing  this  course,  how  well  would  you  expect  to  be  able  to 
perfora  each  of  the  following  activities? 

A.  I  expect  to  do  this  very  easily. 

B.  I  expect  to  do  this  easl ly. 

C.  1  expect  to  do  this  with  difficulty. 

D.  I  expect  to  do  this  with  great  difficulty. 

_  21.  Hall  a  taxi,  give  destination,  pay  fare  and  tip. 

_  22.  Engage  In  "small  talk"  with  Cermans  at  a  social  gathering. 

_  23.  Arrange  hotel  accommodations  for  myself  and  ay  family. 

_  24.  Meet  with  German  military  personnel. 

_  25.  Understand  a  weather  report  In  German  on  the  radio. 

_  26.  Read  and  understand  official  correspondence  in  German. 

_  27.  Make  a  welcoming  or  farewell  speech  In  German  to  a  German 

audience. 

_  28.  Make  an  official  phone  call  In  German. 

_  29.  Meet  with  a  local  government  official. 

_  30.  Respond  to  a  complaint  by  the  German  police. 
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Entry  Proficiency  Exam  (EPE) 

p-  Pronunciation;  F-  Fluency;  S/C-  Sociollgulstic  Cultural; 

C-  Crammer;  V-  Vocabulary;  T-  Tasking 

Assessment  Iteas 

P  P  S/C  C  V  T 


1.  Ask  candidate  to  naae  basic  objects  In  the 
rooa.  (table,  chair,  wall,  floor  door,  window, 
light /lamp,  plant,  picture) 

2.  Ask  candidate  about  the  colors  of  the  above- 
mentioned  objects  (white,  black,  gray,  red, 
green,  yellow,  blue,  brown) 

3.  Ask  about  family/family  members  (father, 
mother,  children,  baby,  brother,  sister, 
grandmother,  grandfather 

4.  Ask  about  the  weather  (is  it  hot,  cold, 
warm,  windy,  raining,  snowing,  nice  weather, 
terrible/bad  weather) 

5.  Ask  about  the  weekdays/S,  M,  T,  V,  TH, 

F,  Sat) 

6.  Ask  about  the  months  (Jan,  Feb,  Mar,  Apr, 
May,  Jun,  Jul,  Aug,  Sept,  Oct,  Nov,  Dec) 

7.  Ask  about  the  day’s  date.  (i.e.  2nd  of 
Jan.  1982) 

8.  Ask  about  the  year  (1984) 

s 

9.  Ask  what  time  it  is 

10.  Do  a  role-play-situation:  bank.  Task: 
cashing  a  traveler's  check;  the  tester-bank 
employee 

11.  Do  a  role-play-situation:  travel  agency. 
Task:  booking  a  flight,  the  tester-agency 
employee 

12.  Do  a  role-play-situation:  on  the  street. 
Task:  giving  direction  to  a  passerby;  tester 
asks. 

13.  Do  a  role-play-situation:  car  breakdown 
Task:  request  help  giving  all  pertinent 
Information. 

14.  Ask  the  student  to  repeat  a  phone  message 
In  the  TL. 
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Appendix  C  Res*,rch  Id<ntl“«ef - 

Post-Course  Questionnaire 

For  each  of  Che  following  activities,  Indicate  which  statement  best  describes 
your  estimation  of  your  ability  to  perfora  the  activity  In  Ceraan.  (Place  the 
appropriate  letter  In  the  space  provided  for  each  Itea). 

a.  I  can  do  this  very  easily. 

b.  1  esn  do  this  easily. 

e.  I  can  do  this  with  difficulty, 

d.  1  can  do  this  with  great  difficulty. 

1.  _  Hall  a  taxi,  give  destination,  pay  are  and  tip. 


2.  _  Engage  In  "small  talk"  with  Germans  at  a  social  gathering. 


3.  _  Arrange  hotel  accommodations  for  myself  and  my  family. 


A.  _  deet  with  German  military  personnel. 

5.  _  Understand  a  weather  report  In  German  on  the  radio. 

6.  _  Read  and  understand  official  correspondance  In  Cerman. 

7.  _  Make  a  welcoming  or  farewell  speech  In  German  to  a  German  audience. 

8.  _  Make  an  official  phone  call  In  Gerasn. 

9.  _  Meet  with  a  local  government  official. 

10.  _  Respond  to  a  complaint  by  the  Genian  police. 

How  effective  was  the  Gerasn  Gateway  course  In  helping  you  learn  each  of  the 
following?  (circle  one) 


11.  Pronunciation 


1 

2 

3 

4 

5 

Extremely 

Extreaely 

Ineffective 

Effective 

12.  Crammar/Syntax 

1 

2 

3 

4 

5 

Extreaely 

Extremely 

Ineffective 

Effective 

13.  Vocabulary 

1 

2 

3 

4 

5 

Extreaely 

Extreaely 

Ineffective 

Effective 

14.  Idiomatic  Expression 

1 

2 

3 

4 

S 

Extreaely 

Extreaely 

Ineffective 

Effective 
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For  each  of  the  language  skills  given  below,  Indicate  how  satisfied  or 
dissatisfied  you  are  with  the  skill  level  you  have  achieved  as  a  result  of  the 
German  Cateway  course,  (circle  one) 


IS.  Understanding  spoken  Geraan  (listening  comprehension) 

1  2  3  4  5 

Extreaely  Extremely 

Dissatisfied  Satisfied 


16.  Conversational  Speaking  Ability 
1  2 
Extremely 
Dissatisfied 


Extremely 

Satisfied 


17.  Ceneral  Reading  Ability  (e.g.,  newspaper,  aenu,  street  signs) 

1  2  3  4  5 

Extremely  Extremely 

Dissatisfied  Satisfied 


How  Important  was  each  of  the  following  Instructional  elements  In  helping  you 
learn  German? 


18.  Tape  Narratlves/Dlalogues 

1  2 

3 

4 

5 

Extremely 

Extreaely 

Unimportant 

Important 

19.  Text  Materials 

1  2 

3 

4 

5 

Extreaely 

Extreaely 

Unimportant 

Important 

20.  Interactive  Video 

1  2 

3 

4 

5 

Extreaely 

Extreaely 

Un Important 

Iaportant 

21.  Other  Audiovisual  Materials 

1  2 

3 

4 

5 

Extreaely 

Extremely 

Unimportant 

Iaportant 

22.  Instructora 

1  2 

3 

4 

5 

Extreaely 

Extreaely 

Dnlaportant 

Iaportant 

1  2 

3 

4 

5 

Extreaely 

Extreaely 

Dnlaportant 

Iaportant 

24.  Exercises  at  End  of  Narratives/Dialogues 

1  2 

3 

4 

3 

Extreaely 

Extreaely 

Dnlaportant 
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Iaportant 

25. 


Based  on  your  anticipated  needs,  how  adequate  was  the  aoount  of  material 
in  the  course  which  simulated  real  life  communication? 

1  2  3  4  3 


Extremely 

Inadequate 


Extremely 

Adequate 


26.  Based  on  your  anticipated  needs,  how  adequate  was  the  amount  of  cultural 
Information  provided  In  the  course? 


1  2 

3 

4  5 

Extremely 

Extremely 

Inadequate 

Adequate 

Which  element  of  the  course  did  you  find  most  difficult?  (circle  one) 

a.  Listening  comprehension 

d. 

Vocabulary 

b.  Pronunciation 

e. 

Reading  comprehension 

c.  Craramar/syntax 

f. 

Conversational  fluency 

Which  teachlng/learnlng  component 
marked  In  the  question  26? 

was 

most  useful  In  learning  what  you 

a.  Tapes 

d. 

Tests 

b.  Texts 

e. 

Interactive  Video 

c.  Instructors 

f. 

Other  audiovisual  materials 

29.  Which  element  of  the  course  did  you  find  the  eaaleat?  (circle  one) 

a.  Listening  comprehension  d.  Vocabulary 

b.  Pronunciation  e.  Reading  comprehension 

c.  Grammar/syntax  f.  Conversational  fluency 

30.  Which  teachlng/learnlng  component  was  most  useful  In  learning  what  you 
narked  In  the  question  28?  (circle  one) 

a.  Tapes  d.  Tests 

b.  Texts  e«  Interactive  Video 

c.  Instructors  f.  Other  Interactive  Video 


31.  Bow  would  you  describe  your  experience  of  the  German  Gateway  course 

without  the  Interactive  video  compared  to  your  experiences  of  the  course 
with  Interactive  video? 

a.  I  liked  non-video  better;  I  felt  I  learned  more  with  that  approach. 

b.  I  reacted  to  the  two  approaches  about  the  ssme. 

c.  I  enjoyed  the  video  more,  but  ay  learning  was  about  the  same. 

d.  I  preferred  the  video,  and  It  seemed  to  Improve  ay  acquisition  and  use 
of  German. 

e.  I  did  not  have  sufficient  opportunity  to  use  the  Interactive  video  to 
make  a  comparison. 


32.  If  you  marked  ja  or  d^  in  the  previous  question.  Indicate  In  what  area(s) 
your  preferred  appToach- was  superior.  (You  may  circle  more  than  one.) 


a.  Listening  comprehension 

b.  Pronunciation 

c.  Graamar/syntax 

d.  Vocabulary 

e.  keadlng  comprehension 

f.  Conversation  fluency 


h. 


1. 


Feeling  for  sltnatlooal 
appropriateness 
Confidence  In  the  language  and 
likelihood  of  trying  to  use  Geratn 
Understanding  of  tbs  German  setting 
or  culture 

Motivation  or  enjoyment 
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33.  Old  you  have  any  probleas  with  the  equipment  used  In  this  course  (e.g., 
poor  fidelity  of  tapes;  tape  player  aalfunctlon,  etc.)? 

a.  Yes 


specify 

b.  No 

34.  Did  you  have  any  probleas  with  the  content  of  the  taped  material  used  In 
this  course  (e.g.,  sounds  gsrbled/slurred,  speech  too  fast  or  Inaudible, 
etc.) 

a.  Yes 


specify 

3S.  Did  you  use  the  Headstart  video  aaterlals? 

a.  Yes 

b.  No 

If  no,  explain  _ 


36.  During  the  course  of  studies,  did  your  Interest  In  the  prograa: 

a.  Increase 

b.  Remain  about  the  same 

c.  Decrease 

why  _ 


37.  Did  you  find  the  6  weeks  allowed  for  the  Gateway  Prograa  to  be:  (circle 
one) 

a.  Par  too  short  a  time 

b.  Too  short  a  time 

c.  Just  right 

d.  Too  long 

e.  Psr  too  long 

38.  How  many  weeks  would  have  suited  you  best?  _______ 

39.  How  do  you  think  the  course  could  have  been  laproved? 


Appendix  D 

INTERACTIVE  VIDEODISC  USE  QUESTIONNAIRE 


The  questions  which  follow  deal  with  the  use  of  the  interactive  video  In 
teaching  a  foreign  language.  As  a  first  user  of  these  materials  we  would  like 
your  reactiors,  and  any  additional  comments  you  wish  to  make,  regarding  this 
method  of  training. 


Listed  below  are  some  characteristics  of  Interactive  video  which  students  and 
teachers  have  found  to  be  significant  aids  in  learning.  How  Important  would 
you  say  each  of  these  characteristics  was  to  you  in  learning  German 
(circle  one). 


Very 

Unimportant 


Very 

Important 


1.  Immediate  feedback 

2.  Visual  presentation 


12  3  4  5 

12  3  4  5 


3.  Exposure  to  a  variety  of  German  speakers  1  2  3  4  5 

4.  Opportunity  for  contextual  learning  1234  5 

(i.e.,  material  presented  in  context  of 

real  situations) 


5.  Active  Involvement  1234  5 

(i.e.,  requirement  to  respond  in  order  to 

continue) 

6.  Interactivity  1234  5 

(i.e.,  content  responsive  to  student 

choice) 

How  useful  did  you  find  each  of  the  following  features  of  the  Interactive  video? 


7.  Select  sub-segment  while  watching 
Tell  (Return) 


Use! 


ess 


Very 

Useful 


2  3  4  5 


8.  Replay  video  fF1) 

9.  Keyword/phrase  helps  (F2) 

10.  Text  with  audio  (F3,1) 

11.  Vocabulary  helps  (F3t2) 

12.  Gramar /Cultural  Notes  (F3t3) 

13.  Questions  (F4) 

14.  Ability  to  use  help  features  while 
working  with  questions 


2  3  4  5 

2  3  4  5 

2  3  4  5 

2  3  4  5 

2  3  4  5 

2  3  4  5 

2  3  4  5 


D-l 


15.  Old  you  experience  any  physical  discomfort  such  as  eye  fatigue,  headaches, 
or  lower  Sack  pain  resulting  from  your  use  of  the  interactive  video 
(circle  cne) . 

1.  No  (If  no,  go  to  question  19) 

2.  Yes  _ 

describe 

If  yes, 

16.  What  effect  did  the  problem  have  on  your  continued  use  of  the 
interactive  video? 

1 .  none 

2.  somewhat  reduced  my  use  of  interactive  video 

3.  greatly  reduced  my  use  of  interactive  video 

4.  ended  my  use  of  interactive  video 

5.  other  _ 


17.  How  frequently,  did  you  experience  the  physical  problem(s)  when  using 
interactive  video  (circle  one). 

1.  all  the  time 

2.  over  half  the  time 

3.  about  half  the  time 

4.  less  than  half  the  time 

5.  very  Infrequently 

18.  Approximately  how  long  could  you  work  with  the  interactive  video 

before  experiencing  symptoms?  _ 


1).  Seme  Interactive  video  exercises  were  presented  via  screen  text  and  some 
via  audio.  Which  do  you  prefer?  (check  only  one) 

_  I  preferred  the  video  exercises  because  _ 


I  preferred  the  text  exercises  because 


I  found  them  equally  effective  and  prefer  the  variation. 
I  have  no  preference. 

Other  (explain)  _ 


In  numbers  20-28,  circle  the  response  which  best  describes  your  behavior  or 
opi nion. 

20.  In  using  the  interactive  video,  did  you  intentionally  select  incorrect 
answers  in  order  to  review/learn  from  the  various  help  features? 


Never 

Seldom 

Sometimes 

Often 

Regularly 

1 

2 

3 

4 

5 

When  you  made  a 
other  paths  to 

choice 

see  the 

in  a  video  segment,  did  you  go  back  and 
difference? 

follow 

Never 

Seldom 

Sometimes  Often 

Regularly 

1 

2 

3  4 

5 

22.  How  often  were  you  able  to  grasp  the  general  meaning  of  the  video 
sequences  on  the  first  presentation? 

Never  Seldom  Sometimes  Often  Regularly 

12  3  *  5 


2j.  How  interesting  did  you  find  the  interactive  video  materials  to  be? 

12  3  »  5 

Very  Uninteresting  Some  Interesting  Interesting  Very 

Uninteresting  Some  Uninteresting  Interesting 
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2«.  Hew  easy  or  difficult  did  you  find  use  ef  the  Interactive  video  equipment 
to  be? 


1 

2 

3  4 

5 

Very  Difficult 

Difficult 

Borderline  Easy 

Very  Easy 

to  Use 

to  Use 

to  Use 

to  Use 

i.  How  adequate  was  the  instruction  you  received  on  how  to  use 

the 

interactive  video  equipment 

(e.g.,  you  usually  knew  what 

to 

do)? 

1 

2 

3  4 

5 

Totally 

Barely 

Adequate  Quite 

Completely 

Inadequate 

Adequate 

Adequate 

Adequate 

i.  How  often  were  you  able  to  use  the  interactive  video  when 

you  wanted 

(i.e.,  hours 

available,  sufficient  number  of  units)? 

1 

2 

3  4 

5 

Almost 

Seldom 

Sometimes  Usually 

Always 

Never 

.  Would  you  recommend  continued  use  of  interactive  video  in 

the  German 

Gateway  Program? 

1 

2 

3  4 

5 

Strongly 

Recommend 

Unsure  Recommend 

Strongly 

Recommend 

Against 

Recommend 

Against 

.  How  often  did 

you  use  the  interactive  video  materials? 

1 

2 

3  4 

5 

Never 

Seldom 

Sometimes  Frequently 

Dally 

How  Important  were  each  of  the  following  factors  to  you  in  using  the  video 
materials  as  much  as  you  did?  (circle  one] 


Unimportant 


29.  Learn  better  with  visuals  1  2  3 

30.  Clarity/quality  of  video  disc  materials  123 

31.  Realistic  context  123 

32.  Like  using  computers  1  2  3 

33.  Quality  of  equipment  1  2  3 

34.  Other  (explain) 


Important 
4  5 

4  5 

4  5 

4  5 

4  5 
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He*  Important  were  each  of  the  following  factors  to  you  in  not  using  the 
Interactive  video  mere  than  you  did? 


35.  Learn  better  from  other  materials 

Unimportant 

1  2 

3 

4 

Important 

5 

36.  Not  useful 

to  pass  tests 

1 

2 

3 

4 

5 

37.  Procedures 

distracted  from  learning 

1 

2 

3 

4 

5 

38.  Not  enough 

time 

1 

2 

3 

4 

5 

39.  Don’t  like 

using  computers 

1 

2 

3 

4 

5 

40 .  Other  (explain)  _ 

i*1.  Hew  do  you  think  the  interactive  video  would  be  most  effectively  used  in 
the  German  Gateway  Program? 


a.  As  a  required  supplement  to  the  audio  tapes  and  text 
materials. 

b.  As  a  replacement  for  the  audio  tapes  with  text  materials 
developed  to  accompany  it. 

c.  As  an  available  supplement  tc  the  structure  guide  and 
workbook  material  for  those  who  are  interested  and  have  the 
time  to  use  it. 

d .  Other  _ _ 

describe 


42.  Based  on  your  experience,  how  would  you  reconend  someone  coming  Into  the 
program  distribute  their  time  (other  than  with  instructor)  in  using  course 
materials  in  order  to  benefit  the  most  from  this  program? 

a.  audio  tapes _ 1 

b.  texts  _____  i 

c.  interactive  video _ J 

d.  structure  gulde/wcrkbook  % 

43.  What  Improvements/changes  would  you  suggest  in  the  interactive  video 
materials? 
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«a.  Dc  you  have  ary  recemmendat  lor  for  charges  Ir  the  set-up  cf  the 

irteractive  video  study  statlors  which  would  sake  then  more  corverlert? 


05.  Did  you  develop  any  particular  techniques,  strategies,  short-cuts,  review 
procedures  that  you  found  especially  effective  in  your  use  of  the 
Interactive  video?  If  so,  briefly  describe.  (Use  bottom  or  back  of  this 
page  If  you  need  more  room.) 


46.  Which  statement  below  best  describes  your  overall  reaction  to  the  use  of 
interactive  video  in  foreign  language  Instruction?  (circle  one) 

1.  Excellent  innovation;  great  potential. 

2.  Useful  aid;  significantly  contributes  to  learning. 

3.  Some  possible  benefit  to  learning. 

4.  Minor  benefit;  contributes  little  to  learning. 

5.  Useless;  a  waste  of  time,  effort  and  money. 
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Appendix  E 

ORAL  PROFICIENCY  EXAMINATION 

1)  Warm  und  wo  sind  ale  geboren? 

Pronunuation  Fluency  Sociolingustic/Culture  Grammar 
0-5  -  -  0-5 

Vocabulary  Task  TOTAL 

0-5  0-5  20 

2)  Sprechen  Sle  bltte  uber  das  Wetter  In  Monterey. 

P  F  S/C  G  V  T  TOTAL 

0-5  0-5  -  0-5  0-10  0-10  35 

3)  Was  machen  Sle  melstens  am  Wochenende? 

P  F  S/C  G  V  T  TOTAL 

0-5  0-5  -  0-10  0-15  0-10  H5 

A)  Your  gateway  group  is  holding  it's  annual  reunion  in 
January.  It  is  your  task  to  phone  a  restaurant  and  ask  about 
opening  and  closing  times,  the  type  of  food  the  restaurant  serves 
and  the  prices.  Then  make  reservations  for  the  group,  giving  all 
the  usual  information  for  this  task. 

P  F  S/C  G  V  T  TOTAL 

0-5  0-5  0-5  0-10  0-20  0-20  65 

5)  You  are  at  a  car  rental  agency.  Communicate  your  wishes  in 
regards  to  a  rental.  You  might  mention  the  type  of  car  you  want, 
the  duration  of  the  rental,  point  of  return,  insurance  etc. 

P  F  S/C  G  V  T  TOTAL 

0-5  0-5  0-5  0-10  0-20  0-20  65 
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6)  You  are  at  the  reception  desk  of  a  German  hotel.  Tell  the 
clerk  what  kind  of  room  you  want;  then  ask  about  the  price, 
whether  breakfast  Is  Included,  where  you  can  park  your  car,  etc. 


P 

P 

S/C 

G 

V 

T 

TOTAL 

0-5 

0-5 

0-5 

0-10 

0-20 

0-15 

60 

7) 

Beschrelben 

Sle  bltte  das 

Zimmer 

In  dem 

sle  slch  beflndenl 

P 

P 

S/C 

G 

V 

T 

TOTAL 

0-5 

0-5 

- 

0-10 

0-20 

0-15 

55 

8) 

Erzahlen  Sle 

bltte 

was  Sle 

heute 

morgen 

alles  gemacht  habenl 

P 

P 

S/C 

G 

V 

T 

TOTAL 

0-5 

0-5 

- 

0-20 

0-15 

0-15 

60 

9) 

Bltte 

vergluchen  sle  Monterey  mlt 

lhren 

Helmatstadt ! 

P 

P 

S/C 

G 

V 

T 

TOTAL 

0-5 

0-5 

- 

0-15 

0-15 

0-15 

55 
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